Jul 21 16:28:24.403763 ip-10-0-141-158 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 16:28:24.728331 ip-10-0-141-158 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 16:28:24.728331 ip-10-0-141-158 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 16:28:24.728331 ip-10-0-141-158 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 16:28:24.728331 ip-10-0-141-158 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 16:28:24.728331 ip-10-0-141-158 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 16:28:24.730281 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.730145    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 16:28:24.736625 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736603    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 16:28:24.736625 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736621    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 16:28:24.736625 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736626    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736632    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736636    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736639    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736642    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736645    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736648    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736651    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736653    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736656    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736659    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736661    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736665    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736668    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736680    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736683    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736686    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736688    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736691    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 16:28:24.736773 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736694    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736700    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736703    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736706    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736708    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736711    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736714    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736716    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736719    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736721    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736724    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736726    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736729    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736731    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736734    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736737    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736740    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736743    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736745    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736748    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 16:28:24.737223 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736750    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736753    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736757    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736760    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736764    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736766    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736768    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736771    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736774    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736777    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736780    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736783    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736786    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736788    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736791    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736793    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736796    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736799    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736801    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736804    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 16:28:24.737737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736806    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736809    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736811    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736814    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736816    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736819    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736823    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736826    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736828    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736831    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736833    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736836    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736838    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736841    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736843    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736845    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736848    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736851    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736853    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 16:28:24.738237 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736857    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736859    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736862    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736865    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736867    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736870    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736872    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736875    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736877    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736880    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736883    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736885    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736888    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736890    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736893    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736895    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736898    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736900    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.736903    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737623    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 16:28:24.738717 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737630    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737634    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737637    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737640    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737643    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737646    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737648    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737651    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737654    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737656    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737659    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737664    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737667    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737670    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737673    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737676    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737679    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737682    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737684    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 16:28:24.739231 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737687    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737689    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737692    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737694    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737697    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737700    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737702    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737705    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737707    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737710    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737712    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737714    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737718    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737720    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737723    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737727    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737731    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737733    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737736    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737738    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 16:28:24.739715 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737741    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737744    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737746    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737749    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737752    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737754    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737757    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737759    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737762    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737764    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737768    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737770    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737773    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737775    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737777    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737780    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737783    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737785    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737788    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737790    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 16:28:24.740235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737793    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737796    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737798    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737800    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737803    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737806    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737809    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737811    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737814    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737816    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737819    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737821    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737824    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737828    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737831    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737833    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737836    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737839    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737841    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737844    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 16:28:24.740728 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737846    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737849    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737851    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737854    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737856    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737858    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737861    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737864    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737866    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737869    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737871    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737874    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737876    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737878    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737881    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737883    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737886    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737889    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.737892    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738303    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 16:28:24.741342 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738309    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738312    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738315    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738318    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738321    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738325    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738329    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738332    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738334    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738337    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738340    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738342    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738345    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738347    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738350    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738353    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738355    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738357    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738360    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738362    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 16:28:24.741858 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738365    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738368    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738371    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738373    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738375    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738378    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738382    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738385    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738389    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738392    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738394    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738397    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738400    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738402    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738404    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738417    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738420    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738423    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738425    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 16:28:24.742354 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738428    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738430    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738433    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738435    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738438    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738440    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738443    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738445    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738448    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738451    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738453    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738456    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738458    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738460    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738463    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738465    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738468    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738471    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738474    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738477    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 16:28:24.742866 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738479    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738482    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738485    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738487    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738490    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738492    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738494    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738497    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738499    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738502    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738504    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738507    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738524    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738527    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738529    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738532    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738535    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738537    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738540    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738543    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 16:28:24.743361 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738547    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738550    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738553    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738556    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738558    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738560    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738563    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738565    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738568    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738570    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738574    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738576    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738579    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738582    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738586    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738589    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738592    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738595    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.738598    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738666    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 16:28:24.743859 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738677    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738683    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738687    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738691    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738694    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738698    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738702    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738705    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738708    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738711    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738714    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738717    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738720    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738723    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738726    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738730    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738734    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738741    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738744    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738747    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738750    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738753    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738757    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 16:28:24.744343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738760    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738763    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738766    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738769    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738772    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738775    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738778    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738781    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738785    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738788    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738790    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738793    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738796    2580 flags.go:64] FLAG: --enable-server="true"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738799    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738805    2580 flags.go:64] FLAG: --event-burst="100"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738808    2580 flags.go:64] FLAG: --event-qps="50"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738811    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738813    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738816    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738820    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738822    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738825    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738828    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738831    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738836    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 16:28:24.744906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738844    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738848    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738850    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738853    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738855    2580 flags.go:64] FLAG: --feature-gates=""
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738860    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738863    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738866    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738869    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738872    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738875    2580 flags.go:64] FLAG: --help="false"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738878    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-141-158.ec2.internal"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738881    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738884    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738886    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738889    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738893    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738895    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738898    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738901    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738903    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738906    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738909    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738911    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 21 16:28:24.745650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738914    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738917    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738919    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738922    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738925    2580 flags.go:64] FLAG: --lock-file=""
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738927    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738930    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738934    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738939    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738947    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738950    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738953    2580 flags.go:64] FLAG: --logging-format="text"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738956    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738960    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738963    2580 flags.go:64] FLAG: --manifest-url=""
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738966    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738970    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738973    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738977    2580 flags.go:64] FLAG: --max-pods="110"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738979    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738982    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738985    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738987    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738990    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738993    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 16:28:24.746345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.738996    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739002    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739005    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739008    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739011    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739013    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739019    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739022    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739025    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739028    2580 flags.go:64] FLAG: --port="10250"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739030    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739033    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01ca1b455700906f8"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739036    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739038    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739042    2580 flags.go:64] FLAG: --register-node="true"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739045    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739048    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739056    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739059    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739062    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739066    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739071    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739073    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739076    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739079    2580 flags.go:64] FLAG: --runonce="false"
Jul 21 16:28:24.746985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739081    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739084    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739087    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739090    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739092    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739095    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739098    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739101    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739103    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739106    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739108    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739111    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739113    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739116    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739119    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739124    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739126    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739129    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739137    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739140    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739142    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739147    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739149    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739152    2580 flags.go:64] FLAG: --v="2"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739156    2580 flags.go:64] FLAG: --version="false"
Jul 21 16:28:24.747623 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739166    2580 flags.go:64] FLAG: --vmodule=""
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739169    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.739172    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739315    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739319    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739322    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739325    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739328    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739330    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739333    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739335    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739338    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739340    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739343    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739345    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739348    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739351    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739353    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739355    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739358    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 16:28:24.748228 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739360    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739363    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739365    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739368    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739370    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739372    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739375    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739379    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739384    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739386    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739389    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739392    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739394    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739404    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739406    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739410    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739412    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739415    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739417    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739420    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 16:28:24.748747 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739422    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739424    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739427    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739429    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739432    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739434    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739437    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739440    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739442    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739445    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739447    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739450    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739452    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739454    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739457    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739459    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739462    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739464    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739467    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739469    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 16:28:24.749235 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739473    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739476    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739478    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739480    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739483    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739485    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739493    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739497    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739499    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739502    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739505    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739507    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739524    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739527    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739529    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739532    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739536    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739539    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739542    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 16:28:24.749776 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739545    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739547    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739550    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739552    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739556    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739559    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739562    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739564    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739567    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739569    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739572    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739574    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739576    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739580    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739583    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739585    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739587    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739596    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739598    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 16:28:24.750238 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739601    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739610    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739613    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.739615    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.740164    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.746926    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.746940    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.746962    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.746972    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747029    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747034    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747037    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747041    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747044    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 16:28:24.750737 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747047    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747050    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747053    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747056    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747059    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747062    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747064    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747067    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747070    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747073    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747075    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747078    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747080    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747083    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747086    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747088    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747091    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747093    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747096    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747098    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 16:28:24.751100 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747101    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747103    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747105    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747107    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747112    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747115    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747117    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747119    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747122    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747124    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747128    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747131    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747134    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747137    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747140    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747143    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747146    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747148    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747151    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747153    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 16:28:24.751647 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747156    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747158    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747160    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747163    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747167    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747170    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747173    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747176    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747178    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747181    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747184    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747186    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747189    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747192    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747195    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747198    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747201    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747204    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747207    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 16:28:24.752132 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747209    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747212    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747214    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747216    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747219    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747221    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747224    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747226    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747229    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747231    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747233    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747236    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747238    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747241    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747243    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747246    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747248    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747251    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747253    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747256    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 16:28:24.752619 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747259    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747263    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747266    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747268    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747271    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747273    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747276    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747279    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747281    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747284    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747287    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747289    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747292    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747295    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747297    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 16:28:24.753135 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.747302    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747414    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747419    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747422    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747424    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747428    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747432    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747434    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747437    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747440    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747442    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747445    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747447    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747450    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747452    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747454    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747457    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747459    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747462    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 16:28:24.753490 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747464    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747466    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747469    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747471    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747474    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747476    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747478    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747481    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747484    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747486    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747489    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747492    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747495    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747497    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747499    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747502    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747505    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747508    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747528    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747531    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 16:28:24.754123 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747534    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747537    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747539    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747542    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747544    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747546    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747549    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747551    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747554    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747556    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747559    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747562    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747564    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747567    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747569    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747571    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747574    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747576    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747579    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747582    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 16:28:24.754825 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747584    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747587    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747589    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747598    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747601    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747603    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747606    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747609    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747611    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747613    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747616    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747618    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747622    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747625    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747628    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747631    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747634    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747637    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747640    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 16:28:24.755541 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747643    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747646    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747648    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747651    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747653    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747656    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747658    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747660    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747663    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747665    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747667    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747670    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747673    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747676    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747678    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747681    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747684    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747687    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747690    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747692    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 16:28:24.756108 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747695    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 16:28:24.756721 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:24.747697    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 16:28:24.756721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.747702    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 16:28:24.756721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.747816    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 16:28:24.756721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.749819    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 16:28:24.756721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.750938    2580 server.go:1019] "Starting client certificate rotation"
Jul 21 16:28:24.756721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.751029    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 16:28:24.756721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.751071    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 16:28:24.756721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.752939    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 16:28:24.756721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.756136    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 16:28:24.764592 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.764502    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 21 16:28:24.766466 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.766450    2580 log.go:25] "Validated CRI v1 image API"
Jul 21 16:28:24.767236 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.767220    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 16:28:24.769948 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.769930    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d0d994c3-b4f7-4710-9704-917e1d4d571a:/dev/nvme0n1p3 e1c41091-312d-455f-97c6-e574806f1a1f:/dev/nvme0n1p4]
Jul 21 16:28:24.770016 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.769948    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 16:28:24.774796 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.774683    2580 manager.go:216] Machine: {Timestamp:2026-07-21 16:28:24.773506826 +0000 UTC m=+0.272868061 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094279 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a8b3b993e81e5953d905d681335a0 SystemUUID:ec2a8b3b-993e-81e5-953d-905d681335a0 BootID:7f55221d-1a29-4382-a788-4d751f87738d Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8d:79:6f:50:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8d:79:6f:50:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:20:13:63:bf:cf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 16:28:24.774796 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.774786    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 16:28:24.774932 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.774860    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 16:28:24.777137 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.777118    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 16:28:24.777291 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.777139    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-158.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 16:28:24.778154 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.778142    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 16:28:24.778190 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.778157    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 16:28:24.778190 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.778164    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 16:28:24.778691 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.778680    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 16:28:24.779796 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.779787    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 16:28:24.780627 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.780611    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 16:28:24.783546 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.783533    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 16:28:24.783661 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.783652    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 16:28:24.783700 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.783682    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 16:28:24.783700 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.783692    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 16:28:24.783753 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.783704    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 16:28:24.783753 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.783713    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 21 16:28:24.783753 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.783733    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 16:28:24.784146 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.784130    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 16:28:24.784206 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.784154    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 16:28:24.785874 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.785859    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 16:28:24.787611 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.787597    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 16:28:24.787650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.787629    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 16:28:24.788696 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788685    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 16:28:24.788737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788702    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 16:28:24.788737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788708    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 16:28:24.788737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788713    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 16:28:24.788737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788726    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 16:28:24.788737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788735    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 16:28:24.788866 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788747    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 16:28:24.788866 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788752    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 16:28:24.788866 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788758    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 16:28:24.788866 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788764    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 16:28:24.788866 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788775    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 16:28:24.788866 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.788783    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 16:28:24.789337 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.789324    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 16:28:24.789370 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.789339    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 16:28:24.793950 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.793916    2580 server.go:1268] "Started kubelet"
Jul 21 16:28:24.794033 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.793964    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 16:28:24.794153 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.794105    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 16:28:24.794246 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.794171    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 16:28:24.796140 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.796112    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-158.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 16:28:24.796259 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.796234    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 16:28:24.796344 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.796241    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-158.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 16:28:24.796285 ip-10-0-141-158 systemd[1]: Started Kubernetes Kubelet.
Jul 21 16:28:24.797305 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.796315    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 16:28:24.797305 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.797095    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 16:28:24.798737 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.798721    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 16:28:24.801256 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.801242    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l4n8r"
Jul 21 16:28:24.802452 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.801696    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-158.ec2.internal.18c45ae7fe2c5c3d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-158.ec2.internal,UID:ip-10-0-141-158.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-158.ec2.internal,},FirstTimestamp:2026-07-21 16:28:24.793824317 +0000 UTC m=+0.293185552,LastTimestamp:2026-07-21 16:28:24.793824317 +0000 UTC m=+0.293185552,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-158.ec2.internal,}"
Jul 21 16:28:24.803283 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.803269    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 16:28:24.803345 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.803283    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 16:28:24.803971 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.803417    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:24.803971 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.803970    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 16:28:24.804085 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.803986    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 16:28:24.804085 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.803996    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 16:28:24.804179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.804099    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 16:28:24.804179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.804108    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 16:28:24.804179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.804157    2580 factory.go:55] Registering systemd factory
Jul 21 16:28:24.804179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.804171    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 21 16:28:24.804474 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.804415    2580 factory.go:153] Registering CRI-O factory
Jul 21 16:28:24.804474 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.804435    2580 factory.go:223] Registration of the crio container factory successfully
Jul 21 16:28:24.804474 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.804472    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 16:28:24.804474 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.804494    2580 factory.go:103] Registering Raw factory
Jul 21 16:28:24.804776 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.804585    2580 manager.go:1196] Started watching for new ooms in manager
Jul 21 16:28:24.805145 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.805129    2580 manager.go:349] Starting recovery of all containers
Jul 21 16:28:24.811106 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.811084    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l4n8r"
Jul 21 16:28:24.811231 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.811203    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-158.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 16:28:24.811322 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.811301    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 16:28:24.816419 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.816400    2580 manager.go:354] Recovery completed
Jul 21 16:28:24.821595 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.821583    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 16:28:24.823348 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.823331    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 16:28:24.823418 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.823359    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 16:28:24.823418 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.823369    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientPID"
Jul 21 16:28:24.823910 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.823896    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 16:28:24.823910 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.823908    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 16:28:24.823910 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.823910    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 16:28:24.824040 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.823923    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 16:28:24.824551 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.824506    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 16:28:24.824551 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.824546    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 16:28:24.824649 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.824558    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientPID"
Jul 21 16:28:24.825106 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.825037    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-158.ec2.internal.18c45ae7ffeed59e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-158.ec2.internal,UID:ip-10-0-141-158.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-158.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-158.ec2.internal,},FirstTimestamp:2026-07-21 16:28:24.82334659 +0000 UTC m=+0.322707831,LastTimestamp:2026-07-21 16:28:24.82334659 +0000 UTC m=+0.322707831,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-158.ec2.internal,}"
Jul 21 16:28:24.825897 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.825886    2580 policy_none.go:49] "None policy: Start"
Jul 21 16:28:24.825948 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.825901    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 16:28:24.825948 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.825911    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 16:28:24.827849 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.827834    2580 policy_none.go:47] "Start"
Jul 21 16:28:24.862158 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.862018    2580 manager.go:337] "Starting Device Plugin manager"
Jul 21 16:28:24.862241 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.862183    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 16:28:24.862241 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.862197    2580 server.go:87] "Starting device plugin registration server"
Jul 21 16:28:24.862437 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.862424    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 16:28:24.862501 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.862440    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 16:28:24.862575 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.862552    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 16:28:24.862680 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.862635    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 16:28:24.862729 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.862681    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 16:28:24.863079 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.863059    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 16:28:24.863158 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.863109    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:24.932175 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.932150    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 16:28:24.934321 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.933364    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 16:28:24.934321 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.933395    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 16:28:24.934321 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.933414    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 16:28:24.934321 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.933441    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 16:28:24.935760 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.935746    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 16:28:24.963567 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.963550    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 16:28:24.966467 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.966453    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 16:28:24.966546 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.966480    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 16:28:24.966546 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.966489    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientPID"
Jul 21 16:28:24.966546 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.966529    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-158.ec2.internal"
Jul 21 16:28:24.972445 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:24.972430    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-158.ec2.internal"
Jul 21 16:28:24.972503 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.972452    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-158.ec2.internal\": node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:24.983833 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:24.983785    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:25.034258 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.034239    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal"]
Jul 21 16:28:25.034324 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.034295    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 16:28:25.035066 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.035051    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 16:28:25.035111 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.035081    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 16:28:25.035111 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.035092    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientPID"
Jul 21 16:28:25.037289 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.037277    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 16:28:25.037402 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.037387    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.037462 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.037414    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 16:28:25.037995 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.037977    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 16:28:25.038078 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.037983    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 16:28:25.038078 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.038034    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 16:28:25.038078 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.038045    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientPID"
Jul 21 16:28:25.038078 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.038009    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 16:28:25.038206 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.038090    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientPID"
Jul 21 16:28:25.040246 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.040231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.040334 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.040255    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 16:28:25.041016 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.040998    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 16:28:25.041101 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.041029    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 16:28:25.041101 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.041040    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeHasSufficientPID"
Jul 21 16:28:25.059043 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.059024    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-158.ec2.internal\" not found" node="ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.063540 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.063527    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-158.ec2.internal\" not found" node="ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.084890 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.084872    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:25.105664 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.105642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7558702c6a64ed5a9d091fed6bb3d25-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal\" (UID: \"a7558702c6a64ed5a9d091fed6bb3d25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.105718 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.105672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7558702c6a64ed5a9d091fed6bb3d25-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal\" (UID: \"a7558702c6a64ed5a9d091fed6bb3d25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.105756 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.105703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0609c9a26f287e8753e54b86bb9c45d6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-158.ec2.internal\" (UID: \"0609c9a26f287e8753e54b86bb9c45d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.185691 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.185673    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:25.206054 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.206034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7558702c6a64ed5a9d091fed6bb3d25-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal\" (UID: \"a7558702c6a64ed5a9d091fed6bb3d25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.206117 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.206056    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7558702c6a64ed5a9d091fed6bb3d25-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal\" (UID: \"a7558702c6a64ed5a9d091fed6bb3d25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.206117 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.206071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7558702c6a64ed5a9d091fed6bb3d25-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal\" (UID: \"a7558702c6a64ed5a9d091fed6bb3d25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.206117 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.206098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7558702c6a64ed5a9d091fed6bb3d25-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal\" (UID: \"a7558702c6a64ed5a9d091fed6bb3d25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.206117 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.206113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0609c9a26f287e8753e54b86bb9c45d6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-158.ec2.internal\" (UID: \"0609c9a26f287e8753e54b86bb9c45d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.206252 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.206126    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0609c9a26f287e8753e54b86bb9c45d6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-158.ec2.internal\" (UID: \"0609c9a26f287e8753e54b86bb9c45d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.286400 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.286351    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:25.361769 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.361731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.367047 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.366984    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:25.386541 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.386508    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:25.487103 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.487078    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:25.587596 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.587544    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:25.688098 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.688079    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:25.750524 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.750492    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 16:28:25.750879 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.750655    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 16:28:25.788923 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.788900    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:25.810968 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.810946    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 16:28:25.817139 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.817114    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 16:23:25 +0000 UTC" deadline="2028-04-07 23:27:00.604939656 +0000 UTC"
Jul 21 16:28:25.817139 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:25.817136    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15030h58m34.787806321s"
Jul 21 16:28:25.889440 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.889398    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:25.989712 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:25.989689    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-158.ec2.internal\" not found"
Jul 21 16:28:26.022602 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.022578    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 16:28:26.076992 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:26.076956    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0609c9a26f287e8753e54b86bb9c45d6.slice/crio-bf2bc88df28af3226cbe3311afafb3910fa8d71a350e304e3d42e6b42e91aa06 WatchSource:0}: Error finding container bf2bc88df28af3226cbe3311afafb3910fa8d71a350e304e3d42e6b42e91aa06: Status 404 returned error can't find the container with id bf2bc88df28af3226cbe3311afafb3910fa8d71a350e304e3d42e6b42e91aa06
Jul 21 16:28:26.081193 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.081175    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 16:28:26.100801 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:26.100781    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7558702c6a64ed5a9d091fed6bb3d25.slice/crio-8c6b098a2ff45e6e764ad2f21653c6552e7fb3ce2653ab989a4f1a3a74a6fea3 WatchSource:0}: Error finding container 8c6b098a2ff45e6e764ad2f21653c6552e7fb3ce2653ab989a4f1a3a74a6fea3: Status 404 returned error can't find the container with id 8c6b098a2ff45e6e764ad2f21653c6552e7fb3ce2653ab989a4f1a3a74a6fea3
Jul 21 16:28:26.103665 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.103646    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:26.113253 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.113234    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 16:28:26.114634 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.114620    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal"
Jul 21 16:28:26.122924 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.122910    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 16:28:26.150316 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.149748    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 16:28:26.784902 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.784871    2580 apiserver.go:52] "Watching apiserver"
Jul 21 16:28:26.794586 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.794559    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 16:28:26.795146 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.795124    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-h4rpx","kube-system/konnectivity-agent-gs24x","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9","openshift-cluster-node-tuning-operator/tuned-9w44q","openshift-dns/node-resolver-qg6tx","openshift-image-registry/node-ca-rkqfz","openshift-multus/multus-additional-cni-plugins-7t2fh","openshift-multus/multus-d78c8","kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal","openshift-multus/network-metrics-daemon-k82tw","openshift-network-diagnostics/network-check-target-qfgz6","openshift-network-operator/iptables-alerter-7jlgv","openshift-ovn-kubernetes/ovnkube-node-gn5dd"]
Jul 21 16:28:26.798109 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.798085    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.801791 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.801768    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 16:28:26.801888 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.801829    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-86c6v\""
Jul 21 16:28:26.801994 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.801977    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 16:28:26.802094 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.802077    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 16:28:26.802158 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.802145    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 16:28:26.804133 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.804113    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 16:28:26.804434 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.804414    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 16:28:26.808177 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.808142    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:26.811978 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.811954    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 16:28:26.812190 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.812172    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nnjjm\""
Jul 21 16:28:26.812391 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.812371    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 16:28:26.813254 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.813230    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 16:28:26.813557 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.813506    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:26.814424 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814347    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d103ad70-c070-4a49-842f-c9dacfba853b-iptables-alerter-script\") pod \"iptables-alerter-7jlgv\" (UID: \"d103ad70-c070-4a49-842f-c9dacfba853b\") " pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:26.814424 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2rsh\" (UniqueName: \"kubernetes.io/projected/d103ad70-c070-4a49-842f-c9dacfba853b-kube-api-access-c2rsh\") pod \"iptables-alerter-7jlgv\" (UID: \"d103ad70-c070-4a49-842f-c9dacfba853b\") " pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:26.814629 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-system-cni-dir\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.814629 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-cnibin\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.814629 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b56ec3a5-eb74-4d28-b538-40c09c485e2b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.814629 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4s28\" (UniqueName: \"kubernetes.io/projected/b56ec3a5-eb74-4d28-b538-40c09c485e2b-kube-api-access-w4s28\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.814629 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d103ad70-c070-4a49-842f-c9dacfba853b-host-slash\") pod \"iptables-alerter-7jlgv\" (UID: \"d103ad70-c070-4a49-842f-c9dacfba853b\") " pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:26.814629 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814563    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-os-release\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.814629 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b56ec3a5-eb74-4d28-b538-40c09c485e2b-cni-binary-copy\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.815033 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.815033 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.814665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b56ec3a5-eb74-4d28-b538-40c09c485e2b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.816622 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.815322    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.816622 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.815907    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 16:28:26.816622 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.816212    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 16:28:26.816622 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.816447    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 16:28:26.816878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.816635    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8xhw4\""
Jul 21 16:28:26.819533 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.817834    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:26.819533 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.818505    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5sgcc\""
Jul 21 16:28:26.819533 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.818737    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 16:28:26.819533 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.818951    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 16:28:26.820208 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.820187    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:26.820689 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.820669    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cw8lp\""
Jul 21 16:28:26.820844 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.820704    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 16:28:26.820844 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.820768    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 16:28:26.821528 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.821462    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 16:28:26.822719 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.822698    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 16:28:26.823428 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.823081    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 16:28:26.823428 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.823093    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4w67h\""
Jul 21 16:28:26.823428 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.823295    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 16:28:26.823428 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.823401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.827413 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.827394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:26.827499 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:26.827461    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:26.828729 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.828710    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:26.830608 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.830572    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 16:28:26.830744 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.830572    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 16:28:26.831421 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.831397    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.831879 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.831680    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 16:28:26.831993 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.831896    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 16:28:26.832211 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.832171    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 16:28:26.833222 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.832905    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 16:28:26.833222 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.832939    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mt5b4\""
Jul 21 16:28:26.833575 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.833415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 16:28:26.833575 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.833452    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g7lmg\""
Jul 21 16:28:26.835229 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.835207    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 16:28:26.835543 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.835521    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 16:28:26.835711 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.835682    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s77hd\""
Jul 21 16:28:26.835879 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.835862    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:26.835962 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:26.835927    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:26.837862 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.837823    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:26.837954 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:26.837911    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:26.841502 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.841477    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zflfg"
Jul 21 16:28:26.851246 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.851225    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zflfg"
Jul 21 16:28:26.905547 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.905524    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 16:28:26.915041 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b87439fa-f868-4083-a444-975165f6d0c2-ovn-node-metrics-cert\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.915141 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6gqg\" (UniqueName: \"kubernetes.io/projected/2effe10a-32f9-4b1f-9e73-483f1b165e00-kube-api-access-f6gqg\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:26.915141 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-sys\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.915141 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5fbef28-96e8-46bd-84a7-aa7a08df42d2-host\") pod \"node-ca-rkqfz\" (UID: \"a5fbef28-96e8-46bd-84a7-aa7a08df42d2\") " pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:26.915141 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-slash\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.915364 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-run-netns\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.915364 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-var-lib-openvswitch\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.915364 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-cni-dir\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.915364 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-run-netns\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.915364 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b56ec3a5-eb74-4d28-b538-40c09c485e2b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.915364 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-sys-fs\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:26.915364 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-kubernetes\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.915364 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-lib-modules\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36e0695-ce82-44e8-89fc-689274f87243-tmp-dir\") pod \"node-resolver-qg6tx\" (UID: \"e36e0695-ce82-44e8-89fc-689274f87243\") " pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/77f6aedc-bb21-4183-bce6-420144e434b7-kubelet-config\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-system-cni-dir\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5fbef28-96e8-46bd-84a7-aa7a08df42d2-serviceca\") pod \"node-ca-rkqfz\" (UID: \"a5fbef28-96e8-46bd-84a7-aa7a08df42d2\") " pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq8ld\" (UniqueName: \"kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld\") pod \"network-check-target-qfgz6\" (UID: \"ec9487ce-c019-466f-9731-34e77376169f\") " pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-cni-netd\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/77f6aedc-bb21-4183-bce6-420144e434b7-dbus\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-var-lib-cni-multus\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-run\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e24382da-c768-4aa7-a51e-af3644ed96bf-tmp\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-etc-kubernetes\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-socket-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-systemd-units\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-run-openvswitch\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.915838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2rsh\" (UniqueName: \"kubernetes.io/projected/d103ad70-c070-4a49-842f-c9dacfba853b-kube-api-access-c2rsh\") pod \"iptables-alerter-7jlgv\" (UID: \"d103ad70-c070-4a49-842f-c9dacfba853b\") " pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-modprobe-d\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-registration-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-node-log\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-cni-bin\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5d8448c-ba0c-4893-a7d8-e8e280a32934-cni-binary-copy\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-daemon-config\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-os-release\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b56ec3a5-eb74-4d28-b538-40c09c485e2b-cni-binary-copy\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915883    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e36e0695-ce82-44e8-89fc-689274f87243-hosts-file\") pod \"node-resolver-qg6tx\" (UID: \"e36e0695-ce82-44e8-89fc-689274f87243\") " pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915920    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b56ec3a5-eb74-4d28-b538-40c09c485e2b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.915928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8de554a-353b-4846-bd7f-79b3c8bf0311-agent-certs\") pod \"konnectivity-agent-gs24x\" (UID: \"e8de554a-353b-4846-bd7f-79b3c8bf0311\") " pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916009    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-system-cni-dir\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-device-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:26.916637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-host\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-run-systemd\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-etc-openvswitch\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b87439fa-f868-4083-a444-975165f6d0c2-ovnkube-config\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-cnibin\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-run-k8s-cni-cncf-io\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-cnibin\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916197    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b56ec3a5-eb74-4d28-b538-40c09c485e2b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d103ad70-c070-4a49-842f-c9dacfba853b-host-slash\") pod \"iptables-alerter-7jlgv\" (UID: \"d103ad70-c070-4a49-842f-c9dacfba853b\") " pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np9gb\" (UniqueName: \"kubernetes.io/projected/a5fbef28-96e8-46bd-84a7-aa7a08df42d2-kube-api-access-np9gb\") pod \"node-ca-rkqfz\" (UID: \"a5fbef28-96e8-46bd-84a7-aa7a08df42d2\") " pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-os-release\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916573    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d103ad70-c070-4a49-842f-c9dacfba853b-host-slash\") pod \"iptables-alerter-7jlgv\" (UID: \"d103ad70-c070-4a49-842f-c9dacfba853b\") " pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916650    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-log-socket\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b87439fa-f868-4083-a444-975165f6d0c2-ovnkube-script-lib\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-socket-dir-parent\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-sysctl-conf\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.917244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9xsv\" (UniqueName: \"kubernetes.io/projected/e24382da-c768-4aa7-a51e-af3644ed96bf-kube-api-access-l9xsv\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916769    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b56ec3a5-eb74-4d28-b538-40c09c485e2b-cni-binary-copy\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-kubelet\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q94mx\" (UniqueName: \"kubernetes.io/projected/b87439fa-f868-4083-a444-975165f6d0c2-kube-api-access-q94mx\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-os-release\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-var-lib-kubelet\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twlxd\" (UniqueName: \"kubernetes.io/projected/c5d8448c-ba0c-4893-a7d8-e8e280a32934-kube-api-access-twlxd\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-var-lib-kubelet\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-552nz\" (UniqueName: \"kubernetes.io/projected/e36e0695-ce82-44e8-89fc-689274f87243-kube-api-access-552nz\") pod \"node-resolver-qg6tx\" (UID: \"e36e0695-ce82-44e8-89fc-689274f87243\") " pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.916990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8de554a-353b-4846-bd7f-79b3c8bf0311-konnectivity-ca\") pod \"konnectivity-agent-gs24x\" (UID: \"e8de554a-353b-4846-bd7f-79b3c8bf0311\") " pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-conf-dir\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-sysctl-d\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-systemd\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-var-lib-cni-bin\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.917988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-hostroot\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4s28\" (UniqueName: \"kubernetes.io/projected/b56ec3a5-eb74-4d28-b538-40c09c485e2b-kube-api-access-w4s28\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b56ec3a5-eb74-4d28-b538-40c09c485e2b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-etc-selinux\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b56ec3a5-eb74-4d28-b538-40c09c485e2b-cnibin\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-sysconfig\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hc9n\" (UniqueName: \"kubernetes.io/projected/54b53eb9-2e20-43cc-ad51-f15f75c19947-kube-api-access-8hc9n\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b87439fa-f868-4083-a444-975165f6d0c2-env-overrides\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-system-cni-dir\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-run-multus-certs\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d103ad70-c070-4a49-842f-c9dacfba853b-iptables-alerter-script\") pod \"iptables-alerter-7jlgv\" (UID: \"d103ad70-c070-4a49-842f-c9dacfba853b\") " pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-kubelet-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-tuned\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.917483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-run-ovn\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:26.918539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.918052    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d103ad70-c070-4a49-842f-c9dacfba853b-iptables-alerter-script\") pod \"iptables-alerter-7jlgv\" (UID: \"d103ad70-c070-4a49-842f-c9dacfba853b\") " pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:26.923014 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.922979    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 16:28:26.926750 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.926728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2rsh\" (UniqueName: \"kubernetes.io/projected/d103ad70-c070-4a49-842f-c9dacfba853b-kube-api-access-c2rsh\") pod \"iptables-alerter-7jlgv\" (UID: \"d103ad70-c070-4a49-842f-c9dacfba853b\") " pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:26.926863 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.926760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4s28\" (UniqueName: \"kubernetes.io/projected/b56ec3a5-eb74-4d28-b538-40c09c485e2b-kube-api-access-w4s28\") pod \"multus-additional-cni-plugins-7t2fh\" (UID: \"b56ec3a5-eb74-4d28-b538-40c09c485e2b\") " pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:26.938775 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.938728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal" event={"ID":"a7558702c6a64ed5a9d091fed6bb3d25","Type":"ContainerStarted","Data":"8c6b098a2ff45e6e764ad2f21653c6552e7fb3ce2653ab989a4f1a3a74a6fea3"}
Jul 21 16:28:26.943904 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:26.943883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal" event={"ID":"0609c9a26f287e8753e54b86bb9c45d6","Type":"ContainerStarted","Data":"bf2bc88df28af3226cbe3311afafb3910fa8d71a350e304e3d42e6b42e91aa06"}
Jul 21 16:28:27.018442 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-run\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.018579 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e24382da-c768-4aa7-a51e-af3644ed96bf-tmp\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.018579 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-etc-kubernetes\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.018579 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-socket-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.018579 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-systemd-units\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.018579 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018570    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-run-openvswitch\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-modprobe-d\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-registration-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-node-log\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-cni-bin\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5d8448c-ba0c-4893-a7d8-e8e280a32934-cni-binary-copy\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-daemon-config\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e36e0695-ce82-44e8-89fc-689274f87243-hosts-file\") pod \"node-resolver-qg6tx\" (UID: \"e36e0695-ce82-44e8-89fc-689274f87243\") " pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8de554a-353b-4846-bd7f-79b3c8bf0311-agent-certs\") pod \"konnectivity-agent-gs24x\" (UID: \"e8de554a-353b-4846-bd7f-79b3c8bf0311\") " pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:27.018787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-device-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-host\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-run-systemd\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-etc-openvswitch\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b87439fa-f868-4083-a444-975165f6d0c2-ovnkube-config\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-cnibin\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-run-k8s-cni-cncf-io\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np9gb\" (UniqueName: \"kubernetes.io/projected/a5fbef28-96e8-46bd-84a7-aa7a08df42d2-kube-api-access-np9gb\") pod \"node-ca-rkqfz\" (UID: \"a5fbef28-96e8-46bd-84a7-aa7a08df42d2\") " pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.018988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-log-socket\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b87439fa-f868-4083-a444-975165f6d0c2-ovnkube-script-lib\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-socket-dir-parent\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-sysctl-conf\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9xsv\" (UniqueName: \"kubernetes.io/projected/e24382da-c768-4aa7-a51e-af3644ed96bf-kube-api-access-l9xsv\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.019172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-kubelet\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.019772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.019772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q94mx\" (UniqueName: \"kubernetes.io/projected/b87439fa-f868-4083-a444-975165f6d0c2-kube-api-access-q94mx\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.019772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-os-release\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.019772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-var-lib-kubelet\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.019772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twlxd\" (UniqueName: \"kubernetes.io/projected/c5d8448c-ba0c-4893-a7d8-e8e280a32934-kube-api-access-twlxd\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.019772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-var-lib-kubelet\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.019772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-552nz\" (UniqueName: \"kubernetes.io/projected/e36e0695-ce82-44e8-89fc-689274f87243-kube-api-access-552nz\") pod \"node-resolver-qg6tx\" (UID: \"e36e0695-ce82-44e8-89fc-689274f87243\") " pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:27.019772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8de554a-353b-4846-bd7f-79b3c8bf0311-konnectivity-ca\") pod \"konnectivity-agent-gs24x\" (UID: \"e8de554a-353b-4846-bd7f-79b3c8bf0311\") " pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-conf-dir\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-sysctl-d\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-systemd\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-var-lib-cni-bin\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-hostroot\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-etc-selinux\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-sysconfig\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.019982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hc9n\" (UniqueName: \"kubernetes.io/projected/54b53eb9-2e20-43cc-ad51-f15f75c19947-kube-api-access-8hc9n\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b87439fa-f868-4083-a444-975165f6d0c2-env-overrides\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-system-cni-dir\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-run-multus-certs\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.020096 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-kubelet-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-tuned\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-run-ovn\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b87439fa-f868-4083-a444-975165f6d0c2-ovn-node-metrics-cert\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6gqg\" (UniqueName: \"kubernetes.io/projected/2effe10a-32f9-4b1f-9e73-483f1b165e00-kube-api-access-f6gqg\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-sys\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5fbef28-96e8-46bd-84a7-aa7a08df42d2-host\") pod \"node-ca-rkqfz\" (UID: \"a5fbef28-96e8-46bd-84a7-aa7a08df42d2\") " pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-slash\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020267    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-run-netns\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-var-lib-openvswitch\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-cni-dir\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-run-netns\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-sys-fs\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-kubernetes\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-lib-modules\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36e0695-ce82-44e8-89fc-689274f87243-tmp-dir\") pod \"node-resolver-qg6tx\" (UID: \"e36e0695-ce82-44e8-89fc-689274f87243\") " pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/77f6aedc-bb21-4183-bce6-420144e434b7-kubelet-config\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:27.020699 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5fbef28-96e8-46bd-84a7-aa7a08df42d2-serviceca\") pod \"node-ca-rkqfz\" (UID: \"a5fbef28-96e8-46bd-84a7-aa7a08df42d2\") " pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:27.021455 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8ld\" (UniqueName: \"kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld\") pod \"network-check-target-qfgz6\" (UID: \"ec9487ce-c019-466f-9731-34e77376169f\") " pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:27.021455 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-cni-netd\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.021455 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/77f6aedc-bb21-4183-bce6-420144e434b7-dbus\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:27.021455 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-var-lib-cni-multus\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.021455 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-var-lib-cni-multus\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.021455 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.020684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-run\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.021455 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.021125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-run-systemd\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.021455 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.021157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-run-k8s-cni-cncf-io\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.021455 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.021180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-etc-kubernetes\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.021983 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.021806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-etc-openvswitch\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.023560 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.023527    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-socket-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.023857 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.023834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e24382da-c768-4aa7-a51e-af3644ed96bf-tmp\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.023911 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.023887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-cnibin\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.024150 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.023669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-kubelet\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.024243 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.024219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.024493 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.024473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-os-release\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.024639 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.024539    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-var-lib-kubelet\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.024639 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.024569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b87439fa-f868-4083-a444-975165f6d0c2-ovnkube-config\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.024898 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.024685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-var-lib-kubelet\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.024898 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.024750    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-log-socket\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.024898 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.024812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-sysconfig\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.024898 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.024860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-conf-dir\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.025129 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.024988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-sysctl-d\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.025129 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-systemd\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.025246 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.025143    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:27.025246 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8de554a-353b-4846-bd7f-79b3c8bf0311-konnectivity-ca\") pod \"konnectivity-agent-gs24x\" (UID: \"e8de554a-353b-4846-bd7f-79b3c8bf0311\") " pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:27.025246 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.025230    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs podName:54b53eb9-2e20-43cc-ad51-f15f75c19947 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:27.525202041 +0000 UTC m=+3.024563280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs") pod "network-metrics-daemon-k82tw" (UID: "54b53eb9-2e20-43cc-ad51-f15f75c19947") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:27.025418 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b87439fa-f868-4083-a444-975165f6d0c2-ovnkube-script-lib\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.025418 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025287    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-var-lib-cni-bin\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.025418 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-sys-fs\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.025581 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-hostroot\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.025581 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-sys\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.025581 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025573    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-etc-selinux\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.025784 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b87439fa-f868-4083-a444-975165f6d0c2-env-overrides\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.025784 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-slash\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.025784 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-var-lib-openvswitch\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.025784 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-run-netns\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.025784 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025724    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-system-cni-dir\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.025784 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5fbef28-96e8-46bd-84a7-aa7a08df42d2-host\") pod \"node-ca-rkqfz\" (UID: \"a5fbef28-96e8-46bd-84a7-aa7a08df42d2\") " pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:27.025784 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025780    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-run-multus-certs\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-cni-dir\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-host-run-netns\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-kubelet-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025895    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-socket-dir-parent\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-run-ovn\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.025950    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/77f6aedc-bb21-4183-bce6-420144e434b7-kubelet-config\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.026009    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026010    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-kubernetes\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.026061    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret podName:77f6aedc-bb21-4183-bce6-420144e434b7 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:27.526038036 +0000 UTC m=+3.025399263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret") pod "global-pull-secret-syncer-h4rpx" (UID: "77f6aedc-bb21-4183-bce6-420144e434b7") : object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-systemd-units\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.026179 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026158    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-lib-modules\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.026731 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-node-log\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.026731 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-modprobe-d\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.026731 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-cni-bin\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.026731 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-registration-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.026731 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026161    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-run-openvswitch\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.026731 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.026731 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36e0695-ce82-44e8-89fc-689274f87243-tmp-dir\") pod \"node-resolver-qg6tx\" (UID: \"e36e0695-ce82-44e8-89fc-689274f87243\") " pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:27.027022 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5fbef28-96e8-46bd-84a7-aa7a08df42d2-serviceca\") pod \"node-ca-rkqfz\" (UID: \"a5fbef28-96e8-46bd-84a7-aa7a08df42d2\") " pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:27.027022 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b87439fa-f868-4083-a444-975165f6d0c2-ovn-node-metrics-cert\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.027022 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026977    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e36e0695-ce82-44e8-89fc-689274f87243-hosts-file\") pod \"node-resolver-qg6tx\" (UID: \"e36e0695-ce82-44e8-89fc-689274f87243\") " pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:27.027022 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.026994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c5d8448c-ba0c-4893-a7d8-e8e280a32934-multus-daemon-config\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.027176 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.027054    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b87439fa-f868-4083-a444-975165f6d0c2-host-cni-netd\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.027176 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.027122    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/77f6aedc-bb21-4183-bce6-420144e434b7-dbus\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:27.027244 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.027184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-host\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.027298 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.027241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2effe10a-32f9-4b1f-9e73-483f1b165e00-device-dir\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.027498 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.027476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5d8448c-ba0c-4893-a7d8-e8e280a32934-cni-binary-copy\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.027558 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.021230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-sysctl-conf\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.030287 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.030266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e24382da-c768-4aa7-a51e-af3644ed96bf-etc-tuned\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.031899 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.031873    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8de554a-353b-4846-bd7f-79b3c8bf0311-agent-certs\") pod \"konnectivity-agent-gs24x\" (UID: \"e8de554a-353b-4846-bd7f-79b3c8bf0311\") " pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:27.035914 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.035860    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 16:28:27.035914 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.035889    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 16:28:27.035914 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.035903    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jq8ld for pod openshift-network-diagnostics/network-check-target-qfgz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:27.036130 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.035977    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld podName:ec9487ce-c019-466f-9731-34e77376169f nodeName:}" failed. No retries permitted until 2026-07-21 16:28:27.535945878 +0000 UTC m=+3.035307123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jq8ld" (UniqueName: "kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld") pod "network-check-target-qfgz6" (UID: "ec9487ce-c019-466f-9731-34e77376169f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:27.038369 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.038279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twlxd\" (UniqueName: \"kubernetes.io/projected/c5d8448c-ba0c-4893-a7d8-e8e280a32934-kube-api-access-twlxd\") pod \"multus-d78c8\" (UID: \"c5d8448c-ba0c-4893-a7d8-e8e280a32934\") " pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.038465 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.038363    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hc9n\" (UniqueName: \"kubernetes.io/projected/54b53eb9-2e20-43cc-ad51-f15f75c19947-kube-api-access-8hc9n\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:27.040396 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.040368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q94mx\" (UniqueName: \"kubernetes.io/projected/b87439fa-f868-4083-a444-975165f6d0c2-kube-api-access-q94mx\") pod \"ovnkube-node-gn5dd\" (UID: \"b87439fa-f868-4083-a444-975165f6d0c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.040489 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.040416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9xsv\" (UniqueName: \"kubernetes.io/projected/e24382da-c768-4aa7-a51e-af3644ed96bf-kube-api-access-l9xsv\") pod \"tuned-9w44q\" (UID: \"e24382da-c768-4aa7-a51e-af3644ed96bf\") " pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.040917 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.040896    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np9gb\" (UniqueName: \"kubernetes.io/projected/a5fbef28-96e8-46bd-84a7-aa7a08df42d2-kube-api-access-np9gb\") pod \"node-ca-rkqfz\" (UID: \"a5fbef28-96e8-46bd-84a7-aa7a08df42d2\") " pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:27.040917 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.040904    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-552nz\" (UniqueName: \"kubernetes.io/projected/e36e0695-ce82-44e8-89fc-689274f87243-kube-api-access-552nz\") pod \"node-resolver-qg6tx\" (UID: \"e36e0695-ce82-44e8-89fc-689274f87243\") " pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:27.042107 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.042085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6gqg\" (UniqueName: \"kubernetes.io/projected/2effe10a-32f9-4b1f-9e73-483f1b165e00-kube-api-access-f6gqg\") pod \"aws-ebs-csi-driver-node-svhm9\" (UID: \"2effe10a-32f9-4b1f-9e73-483f1b165e00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.115304 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.115272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7t2fh"
Jul 21 16:28:27.128877 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.128692    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7jlgv"
Jul 21 16:28:27.139710 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.139685    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9"
Jul 21 16:28:27.146704 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.146686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9w44q"
Jul 21 16:28:27.155696 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.155676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qg6tx"
Jul 21 16:28:27.163605 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.163575    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rkqfz"
Jul 21 16:28:27.173482 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.173459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:27.180474 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.180455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:27.187556 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.187517    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d78c8"
Jul 21 16:28:27.347084 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.346985    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 16:28:27.526116 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.526079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:27.526308 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.526227    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:27.526308 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.526296    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs podName:54b53eb9-2e20-43cc-ad51-f15f75c19947 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:28.526274383 +0000 UTC m=+4.025635629 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs") pod "network-metrics-daemon-k82tw" (UID: "54b53eb9-2e20-43cc-ad51-f15f75c19947") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:27.627147 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.627061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8ld\" (UniqueName: \"kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld\") pod \"network-check-target-qfgz6\" (UID: \"ec9487ce-c019-466f-9731-34e77376169f\") " pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:27.627147 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.627111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:27.627380 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.627247    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:27.627380 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.627273    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 16:28:27.627380 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.627299    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 16:28:27.627380 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.627313    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jq8ld for pod openshift-network-diagnostics/network-check-target-qfgz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:27.627380 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.627315    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret podName:77f6aedc-bb21-4183-bce6-420144e434b7 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:28.627295786 +0000 UTC m=+4.126657010 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret") pod "global-pull-secret-syncer-h4rpx" (UID: "77f6aedc-bb21-4183-bce6-420144e434b7") : object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:27.627380 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:27.627375    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld podName:ec9487ce-c019-466f-9731-34e77376169f nodeName:}" failed. No retries permitted until 2026-07-21 16:28:28.627356074 +0000 UTC m=+4.126717356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jq8ld" (UniqueName: "kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld") pod "network-check-target-qfgz6" (UID: "ec9487ce-c019-466f-9731-34e77376169f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:27.855221 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.855180    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 16:23:27 +0000 UTC" deadline="2028-03-04 22:50:01.469837722 +0000 UTC"
Jul 21 16:28:27.855221 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:27.855212    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14214h21m33.614629591s"
Jul 21 16:28:28.185396 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:28.185363    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87439fa_f868_4083_a444_975165f6d0c2.slice/crio-708a55ddb878a7090a5a3aa5261d78cdee21f74287100a153353aa9fbe1f17e1 WatchSource:0}: Error finding container 708a55ddb878a7090a5a3aa5261d78cdee21f74287100a153353aa9fbe1f17e1: Status 404 returned error can't find the container with id 708a55ddb878a7090a5a3aa5261d78cdee21f74287100a153353aa9fbe1f17e1
Jul 21 16:28:28.186779 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:28.186752    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24382da_c768_4aa7_a51e_af3644ed96bf.slice/crio-1fb79fc0f154b5d981874bff3a692ba63f90edd496a6c5cf73b93b77ff5d650f WatchSource:0}: Error finding container 1fb79fc0f154b5d981874bff3a692ba63f90edd496a6c5cf73b93b77ff5d650f: Status 404 returned error can't find the container with id 1fb79fc0f154b5d981874bff3a692ba63f90edd496a6c5cf73b93b77ff5d650f
Jul 21 16:28:28.187376 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:28.187344    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d8448c_ba0c_4893_a7d8_e8e280a32934.slice/crio-edc486cd15aa9d4fc8a8ffee81af10cd38625a28a389850505e50bf840587490 WatchSource:0}: Error finding container edc486cd15aa9d4fc8a8ffee81af10cd38625a28a389850505e50bf840587490: Status 404 returned error can't find the container with id edc486cd15aa9d4fc8a8ffee81af10cd38625a28a389850505e50bf840587490
Jul 21 16:28:28.190379 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:28.190348    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36e0695_ce82_44e8_89fc_689274f87243.slice/crio-8f3b9e61c007bac0694f1db6ba2a040bb7c459bda47075eb651db6f2aceec76e WatchSource:0}: Error finding container 8f3b9e61c007bac0694f1db6ba2a040bb7c459bda47075eb651db6f2aceec76e: Status 404 returned error can't find the container with id 8f3b9e61c007bac0694f1db6ba2a040bb7c459bda47075eb651db6f2aceec76e
Jul 21 16:28:28.192032 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:28:28.192009    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8de554a_353b_4846_bd7f_79b3c8bf0311.slice/crio-6a751844ff1a484a9ef9a0bfcde8b8eac40abc294f0f557a965c7c731a67c4a6 WatchSource:0}: Error finding container 6a751844ff1a484a9ef9a0bfcde8b8eac40abc294f0f557a965c7c731a67c4a6: Status 404 returned error can't find the container with id 6a751844ff1a484a9ef9a0bfcde8b8eac40abc294f0f557a965c7c731a67c4a6
Jul 21 16:28:28.534021 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.533842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:28.534021 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.534009    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:28.534182 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.534072    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs podName:54b53eb9-2e20-43cc-ad51-f15f75c19947 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:30.534056015 +0000 UTC m=+6.033417243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs") pod "network-metrics-daemon-k82tw" (UID: "54b53eb9-2e20-43cc-ad51-f15f75c19947") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:28.634845 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.634817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8ld\" (UniqueName: \"kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld\") pod \"network-check-target-qfgz6\" (UID: \"ec9487ce-c019-466f-9731-34e77376169f\") " pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:28.634984 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.634869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:28.635062 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.635010    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:28.635062 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.635013    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 16:28:28.635062 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.635036    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 16:28:28.635062 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.635048    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jq8ld for pod openshift-network-diagnostics/network-check-target-qfgz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:28.635227 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.635077    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret podName:77f6aedc-bb21-4183-bce6-420144e434b7 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:30.635058719 +0000 UTC m=+6.134419960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret") pod "global-pull-secret-syncer-h4rpx" (UID: "77f6aedc-bb21-4183-bce6-420144e434b7") : object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:28.635227 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.635096    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld podName:ec9487ce-c019-466f-9731-34e77376169f nodeName:}" failed. No retries permitted until 2026-07-21 16:28:30.635084516 +0000 UTC m=+6.134445761 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jq8ld" (UniqueName: "kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld") pod "network-check-target-qfgz6" (UID: "ec9487ce-c019-466f-9731-34e77376169f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:28.855693 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.855572    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 16:23:27 +0000 UTC" deadline="2028-01-17 05:14:20.042114038 +0000 UTC"
Jul 21 16:28:28.855693 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.855611    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13068h45m51.186508064s"
Jul 21 16:28:28.943328 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.941770    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:28.943328 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.941953    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:28.943328 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.942474    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:28.943328 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.942646    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:28.943328 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.942740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:28.943328 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:28.942841    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:28.980637 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.980606    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal" event={"ID":"0609c9a26f287e8753e54b86bb9c45d6","Type":"ContainerStarted","Data":"fa784cd72fa21f6552ceaae07422c81b99d065548566a4c81613cc137c3dc6e6"}
Jul 21 16:28:28.988770 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.988737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t2fh" event={"ID":"b56ec3a5-eb74-4d28-b538-40c09c485e2b","Type":"ContainerStarted","Data":"6b9cd7537237917c25277a7bb54b6be8e31fc80054655855fefe8a35dcf5b8f9"}
Jul 21 16:28:28.996639 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.996064    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-158.ec2.internal" podStartSLOduration=2.996044629 podStartE2EDuration="2.996044629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:28:28.995244931 +0000 UTC m=+4.494606182" watchObservedRunningTime="2026-07-21 16:28:28.996044629 +0000 UTC m=+4.495405878"
Jul 21 16:28:28.998306 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:28.998250    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d78c8" event={"ID":"c5d8448c-ba0c-4893-a7d8-e8e280a32934","Type":"ContainerStarted","Data":"edc486cd15aa9d4fc8a8ffee81af10cd38625a28a389850505e50bf840587490"}
Jul 21 16:28:29.005016 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:29.004986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9w44q" event={"ID":"e24382da-c768-4aa7-a51e-af3644ed96bf","Type":"ContainerStarted","Data":"1fb79fc0f154b5d981874bff3a692ba63f90edd496a6c5cf73b93b77ff5d650f"}
Jul 21 16:28:29.009817 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:29.009794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd" event={"ID":"b87439fa-f868-4083-a444-975165f6d0c2","Type":"ContainerStarted","Data":"708a55ddb878a7090a5a3aa5261d78cdee21f74287100a153353aa9fbe1f17e1"}
Jul 21 16:28:29.015937 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:29.015908    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7558702c6a64ed5a9d091fed6bb3d25" containerID="8236520350dc9a6f4cda36f0e88568ad2881c78aa9b0295deed553428d49af9e" exitCode=0
Jul 21 16:28:29.016043 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:29.016003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal" event={"ID":"a7558702c6a64ed5a9d091fed6bb3d25","Type":"ContainerDied","Data":"8236520350dc9a6f4cda36f0e88568ad2881c78aa9b0295deed553428d49af9e"}
Jul 21 16:28:29.030571 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:29.030514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9" event={"ID":"2effe10a-32f9-4b1f-9e73-483f1b165e00","Type":"ContainerStarted","Data":"2e3272ca47c9a8f1ec0463a6f5a80d41ac612b91776ac5604238c9433f14790b"}
Jul 21 16:28:29.036160 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:29.036133    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rkqfz" event={"ID":"a5fbef28-96e8-46bd-84a7-aa7a08df42d2","Type":"ContainerStarted","Data":"250d0858d12c82fcdfc29905fe8a777c27f2a864ab5042b0198e059dbe877363"}
Jul 21 16:28:29.051470 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:29.051442    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7jlgv" event={"ID":"d103ad70-c070-4a49-842f-c9dacfba853b","Type":"ContainerStarted","Data":"162deb0855b3dd8e7b68871202d8bada0ed259570be41151aa10d4427e81be94"}
Jul 21 16:28:29.058410 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:29.058382    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gs24x" event={"ID":"e8de554a-353b-4846-bd7f-79b3c8bf0311","Type":"ContainerStarted","Data":"6a751844ff1a484a9ef9a0bfcde8b8eac40abc294f0f557a965c7c731a67c4a6"}
Jul 21 16:28:29.073804 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:29.073774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qg6tx" event={"ID":"e36e0695-ce82-44e8-89fc-689274f87243","Type":"ContainerStarted","Data":"8f3b9e61c007bac0694f1db6ba2a040bb7c459bda47075eb651db6f2aceec76e"}
Jul 21 16:28:30.087708 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:30.087671    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal" event={"ID":"a7558702c6a64ed5a9d091fed6bb3d25","Type":"ContainerStarted","Data":"7e3005667a381f2d01ce46cecaf40b1ee71b799614351742444f75ac67143bd0"}
Jul 21 16:28:30.104588 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:30.104525    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-158.ec2.internal" podStartSLOduration=4.104503981 podStartE2EDuration="4.104503981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:28:30.103816267 +0000 UTC m=+5.603177517" watchObservedRunningTime="2026-07-21 16:28:30.104503981 +0000 UTC m=+5.603865237"
Jul 21 16:28:30.560080 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:30.559952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:30.560258 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.560128    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:30.560258 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.560191    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs podName:54b53eb9-2e20-43cc-ad51-f15f75c19947 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:34.56017262 +0000 UTC m=+10.059533847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs") pod "network-metrics-daemon-k82tw" (UID: "54b53eb9-2e20-43cc-ad51-f15f75c19947") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:30.660761 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:30.660720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8ld\" (UniqueName: \"kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld\") pod \"network-check-target-qfgz6\" (UID: \"ec9487ce-c019-466f-9731-34e77376169f\") " pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:30.660943 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:30.660782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:30.660943 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.660904    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:30.661052 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.660968    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret podName:77f6aedc-bb21-4183-bce6-420144e434b7 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:34.660948329 +0000 UTC m=+10.160309553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret") pod "global-pull-secret-syncer-h4rpx" (UID: "77f6aedc-bb21-4183-bce6-420144e434b7") : object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:30.661420 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.661398    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 16:28:30.661520 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.661427    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 16:28:30.661520 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.661444    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jq8ld for pod openshift-network-diagnostics/network-check-target-qfgz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:30.661520 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.661502    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld podName:ec9487ce-c019-466f-9731-34e77376169f nodeName:}" failed. No retries permitted until 2026-07-21 16:28:34.661486809 +0000 UTC m=+10.160848034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jq8ld" (UniqueName: "kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld") pod "network-check-target-qfgz6" (UID: "ec9487ce-c019-466f-9731-34e77376169f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:30.934621 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:30.934521    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:30.934798 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.934686    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:30.935107 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:30.935083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:30.935211 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.935188    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:30.935373 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:30.935354    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:30.935527 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:30.935459    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:32.936211 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:32.936177    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:32.936716 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:32.936315    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:32.936803 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:32.936761    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:32.936884 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:32.936871    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:32.936935 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:32.936920    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:32.936976 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:32.936964    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:34.593532 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:34.593466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:34.593944 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.593676    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:34.593944 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.593744    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs podName:54b53eb9-2e20-43cc-ad51-f15f75c19947 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:42.593722229 +0000 UTC m=+18.093083457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs") pod "network-metrics-daemon-k82tw" (UID: "54b53eb9-2e20-43cc-ad51-f15f75c19947") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:34.694614 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:34.694566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8ld\" (UniqueName: \"kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld\") pod \"network-check-target-qfgz6\" (UID: \"ec9487ce-c019-466f-9731-34e77376169f\") " pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:34.694785 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:34.694635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:34.694862 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.694839    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:34.694945 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.694921    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret podName:77f6aedc-bb21-4183-bce6-420144e434b7 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:42.694899239 +0000 UTC m=+18.194260466 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret") pod "global-pull-secret-syncer-h4rpx" (UID: "77f6aedc-bb21-4183-bce6-420144e434b7") : object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:34.695402 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.695379    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 16:28:34.695402 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.695405    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 16:28:34.695635 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.695418    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jq8ld for pod openshift-network-diagnostics/network-check-target-qfgz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:34.695635 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.695471    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld podName:ec9487ce-c019-466f-9731-34e77376169f nodeName:}" failed. No retries permitted until 2026-07-21 16:28:42.695453724 +0000 UTC m=+18.194814969 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jq8ld" (UniqueName: "kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld") pod "network-check-target-qfgz6" (UID: "ec9487ce-c019-466f-9731-34e77376169f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:34.935481 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:34.935402    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:34.935673 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.935528    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:34.935673 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:34.935615    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:34.935791 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.935713    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:34.935791 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:34.935758    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:34.935857 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:34.935837    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:36.935500 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:36.934652    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:36.935500 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:36.934781    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:36.935500 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:36.935111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:36.935500 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:36.935208    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:36.936194 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:36.936165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:36.936316 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:36.936281    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:38.935597 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:38.935509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:38.935597 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:38.935564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:38.936113 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:38.935645    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:38.936113 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:38.935798    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:38.936113 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:38.935839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:38.936113 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:38.935954    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:40.934231 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:40.934196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:40.934744 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:40.934196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:40.934744 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:40.934332    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:40.934744 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:40.934345    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:40.934744 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:40.934434    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:40.934744 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:40.934591    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:42.663759 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:42.663721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:42.664194 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.663881    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:42.664194 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.663954    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs podName:54b53eb9-2e20-43cc-ad51-f15f75c19947 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:58.663935868 +0000 UTC m=+34.163297110 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs") pod "network-metrics-daemon-k82tw" (UID: "54b53eb9-2e20-43cc-ad51-f15f75c19947") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:42.765028 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:42.764997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8ld\" (UniqueName: \"kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld\") pod \"network-check-target-qfgz6\" (UID: \"ec9487ce-c019-466f-9731-34e77376169f\") " pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:42.765194 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:42.765047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:42.765194 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.765152    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:42.765194 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.765183    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 16:28:42.765335 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.765207    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret podName:77f6aedc-bb21-4183-bce6-420144e434b7 nodeName:}" failed. No retries permitted until 2026-07-21 16:28:58.76518928 +0000 UTC m=+34.264550511 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret") pod "global-pull-secret-syncer-h4rpx" (UID: "77f6aedc-bb21-4183-bce6-420144e434b7") : object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:42.765335 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.765210    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 16:28:42.765335 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.765223    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jq8ld for pod openshift-network-diagnostics/network-check-target-qfgz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:42.765335 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.765274    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld podName:ec9487ce-c019-466f-9731-34e77376169f nodeName:}" failed. No retries permitted until 2026-07-21 16:28:58.76525921 +0000 UTC m=+34.264620434 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jq8ld" (UniqueName: "kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld") pod "network-check-target-qfgz6" (UID: "ec9487ce-c019-466f-9731-34e77376169f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:42.934178 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:42.934095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:42.934178 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:42.934129    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:42.934351 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:42.934095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:42.934351 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.934216    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:42.934351 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.934275    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:42.934482 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:42.934373    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:44.935593 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:44.935564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:44.936040 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:44.935664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:44.936040 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:44.935705    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:44.936040 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:44.935706    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:44.936040 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:44.935782    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:44.936040 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:44.935864    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:46.934209 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:46.933732    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:46.934209 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:46.933750    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:46.934209 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:46.933858    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:46.934209 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:46.933992    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:46.934209 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:46.934032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:46.934209 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:46.934135    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:48.128571 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:48.127884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d78c8" event={"ID":"c5d8448c-ba0c-4893-a7d8-e8e280a32934","Type":"ContainerStarted","Data":"b7d5b5c7345af114357d2ff3fa46c6bf3b435d82c8520d263029ffb4a9183226"}
Jul 21 16:28:48.134504 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:48.134116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9w44q" event={"ID":"e24382da-c768-4aa7-a51e-af3644ed96bf","Type":"ContainerStarted","Data":"f073e6fd4e401209dabf4d7e4886985d488958f8c22244e8917b49565a1c2636"}
Jul 21 16:28:48.137323 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:48.136831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9" event={"ID":"2effe10a-32f9-4b1f-9e73-483f1b165e00","Type":"ContainerStarted","Data":"78712cb6e89ec2410980e536f70a2dd1b8686dd1786ce3c672953bf76190e42c"}
Jul 21 16:28:48.139195 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:48.138987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gs24x" event={"ID":"e8de554a-353b-4846-bd7f-79b3c8bf0311","Type":"ContainerStarted","Data":"b7d170fca47a021a303a7640b1e3ad622a91fbf7cb2d110fc47b67fe534ee734"}
Jul 21 16:28:48.144830 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:48.144557    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d78c8" podStartSLOduration=3.353350728 podStartE2EDuration="23.144539183s" totalImagesPullingTime="19.791188455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:28:48.143759986 +0000 UTC m=+23.643121230" watchObservedRunningTime="2026-07-21 16:28:48.144539183 +0000 UTC m=+23.643900423"
Jul 21 16:28:48.162542 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:48.160313    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9w44q" podStartSLOduration=3.365956018 podStartE2EDuration="23.160294449s" totalImagesPullingTime="19.794338431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:28:48.159464533 +0000 UTC m=+23.658825779" watchObservedRunningTime="2026-07-21 16:28:48.160294449 +0000 UTC m=+23.659655695"
Jul 21 16:28:48.178765 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:48.178108    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gs24x" podStartSLOduration=3.447604878 podStartE2EDuration="23.1780871s" totalImagesPullingTime="19.730482222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:28:48.177978428 +0000 UTC m=+23.677339675" watchObservedRunningTime="2026-07-21 16:28:48.1780871 +0000 UTC m=+23.677448345"
Jul 21 16:28:48.934255 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:48.933979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:48.934388 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:48.934314    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:48.934388 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:48.934041    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:48.934489 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:48.934410    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:48.934489 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:48.933979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:48.934597 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:48.934497    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:49.045578 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.045549    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:49.046131 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.046116    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:49.141927 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.141895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rkqfz" event={"ID":"a5fbef28-96e8-46bd-84a7-aa7a08df42d2","Type":"ContainerStarted","Data":"0575ea79b630af03a8c050988e54565b28b2986d1eae4881205e48b05dfc901f"}
Jul 21 16:28:49.143502 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.143479    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qg6tx" event={"ID":"e36e0695-ce82-44e8-89fc-689274f87243","Type":"ContainerStarted","Data":"3edfc0b00ec05c559ce78f7b9aa6dc018c935f0700c57ae6600abd8d5cf2f1d3"}
Jul 21 16:28:49.145105 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.145072    2580 generic.go:356] "Generic (PLEG): container finished" podID="b56ec3a5-eb74-4d28-b538-40c09c485e2b" containerID="04d99d5a1ccad1910232ce3115d2df010d5f0088276552bd4812a487404202fa" exitCode=0
Jul 21 16:28:49.145210 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.145152    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t2fh" event={"ID":"b56ec3a5-eb74-4d28-b538-40c09c485e2b","Type":"ContainerDied","Data":"04d99d5a1ccad1910232ce3115d2df010d5f0088276552bd4812a487404202fa"}
Jul 21 16:28:49.148251 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.148230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd" event={"ID":"b87439fa-f868-4083-a444-975165f6d0c2","Type":"ContainerStarted","Data":"c5534dd19a2551d02c59529659cd9c35d29bcc9af0747b7f1a481e21069d2ce5"}
Jul 21 16:28:49.148324 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.148290    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd" event={"ID":"b87439fa-f868-4083-a444-975165f6d0c2","Type":"ContainerStarted","Data":"ab0422fbbeb5075c603167fbba41952dc2fd6d08289bf0867b81e89dca4a8ea1"}
Jul 21 16:28:49.148370 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.148327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd" event={"ID":"b87439fa-f868-4083-a444-975165f6d0c2","Type":"ContainerStarted","Data":"5a7a163f9500b9295bb761f1f8846a108a13cefba8a60d3357e7f05c7e88c883"}
Jul 21 16:28:49.148409 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.148379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd" event={"ID":"b87439fa-f868-4083-a444-975165f6d0c2","Type":"ContainerStarted","Data":"2a5b88bf0493596da5113292d854b9697a36b440f6ab8ca85d02d99259ff8d48"}
Jul 21 16:28:49.148449 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.148428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd" event={"ID":"b87439fa-f868-4083-a444-975165f6d0c2","Type":"ContainerStarted","Data":"57d232627741ca2a0fd56730505211372da94d1cc0e441ea6ee5ec42cdd28e90"}
Jul 21 16:28:49.148486 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.148475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd" event={"ID":"b87439fa-f868-4083-a444-975165f6d0c2","Type":"ContainerStarted","Data":"b3d5f2ed5d42e06715dbb40390d0ef6e24661e0bd3a666d2c83a8c31c4bfb142"}
Jul 21 16:28:49.148770 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.148753    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:49.149151 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.149136    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gs24x"
Jul 21 16:28:49.180976 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.180937    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rkqfz" podStartSLOduration=4.446169052 podStartE2EDuration="24.180924926s" totalImagesPullingTime="19.734755874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:28:49.158553006 +0000 UTC m=+24.657914249" watchObservedRunningTime="2026-07-21 16:28:49.180924926 +0000 UTC m=+24.680286186"
Jul 21 16:28:49.208249 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.208208    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qg6tx" podStartSLOduration=4.45961469 podStartE2EDuration="24.208197007s" totalImagesPullingTime="19.748582317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:28:49.208014069 +0000 UTC m=+24.707375312" watchObservedRunningTime="2026-07-21 16:28:49.208197007 +0000 UTC m=+24.707558251"
Jul 21 16:28:49.380657 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.380635    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 16:28:49.886650 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.886545    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T16:28:49.380651405Z","UUID":"8abb6ca1-5853-496b-b922-1adbfb4db975","Handler":null,"Name":"","Endpoint":""}
Jul 21 16:28:49.888138 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.888114    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 16:28:49.888272 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:49.888161    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 16:28:50.152754 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:50.152720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9" event={"ID":"2effe10a-32f9-4b1f-9e73-483f1b165e00","Type":"ContainerStarted","Data":"2bf768ec8e04d002360bff8d2c0f1ca000b61615f98aa44bb9e8141c79d5c83c"}
Jul 21 16:28:50.154422 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:50.154398    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7jlgv" event={"ID":"d103ad70-c070-4a49-842f-c9dacfba853b","Type":"ContainerStarted","Data":"0ebd0cbbb9925a83ffb9ac3244caa3cb5f5a0760322fd10481d717c1bbc42812"}
Jul 21 16:28:50.170274 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:50.170230    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7jlgv" podStartSLOduration=5.435469011 podStartE2EDuration="25.170217491s" totalImagesPullingTime="19.73474848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:28:50.170184192 +0000 UTC m=+25.669545440" watchObservedRunningTime="2026-07-21 16:28:50.170217491 +0000 UTC m=+25.669578736"
Jul 21 16:28:50.937665 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:50.937642    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:50.937767 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:50.937682    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:50.937767 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:50.937687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:50.937878 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:50.937763    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:50.937936 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:50.937871    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:50.937990 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:50.937957    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:51.161362 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:51.161327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd" event={"ID":"b87439fa-f868-4083-a444-975165f6d0c2","Type":"ContainerStarted","Data":"242e88c04c94e30bcb8303a3c52b229312acc5bc1bc2d871e149697baf282798"}
Jul 21 16:28:51.163953 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:51.163925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9" event={"ID":"2effe10a-32f9-4b1f-9e73-483f1b165e00","Type":"ContainerStarted","Data":"ad4d822508f6bf80789115fe777c307e502283962aff7d6dcd7ee83e91194e1f"}
Jul 21 16:28:51.181825 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:51.181757    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svhm9" podStartSLOduration=4.074090001 podStartE2EDuration="26.181741155s" totalImagesPullingTime="22.107651154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:28:51.181711612 +0000 UTC m=+26.681072859" watchObservedRunningTime="2026-07-21 16:28:51.181741155 +0000 UTC m=+26.681102397"
Jul 21 16:28:52.938287 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:52.938083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:52.938813 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:52.938083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:52.938813 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:52.938368    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:52.938813 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:52.938459    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:52.938813 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:52.938083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:52.938813 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:52.938582    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:54.172498 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.172317    2580 generic.go:356] "Generic (PLEG): container finished" podID="b56ec3a5-eb74-4d28-b538-40c09c485e2b" containerID="2dbadfe660f736767aff09d12aad7938927f8ae18439dbeb9025b5a2b14f2262" exitCode=0
Jul 21 16:28:54.173288 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.172382    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t2fh" event={"ID":"b56ec3a5-eb74-4d28-b538-40c09c485e2b","Type":"ContainerDied","Data":"2dbadfe660f736767aff09d12aad7938927f8ae18439dbeb9025b5a2b14f2262"}
Jul 21 16:28:54.176557 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.176533    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd" event={"ID":"b87439fa-f868-4083-a444-975165f6d0c2","Type":"ContainerStarted","Data":"9393034bf27cb7e8a31550b8a935ecc65a8bc54e776eb801208fb2d7db1b295d"}
Jul 21 16:28:54.176763 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.176734    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:54.176824 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.176813    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:54.176873 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.176863    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:54.192721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.192701    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:54.192833 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.192813    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:28:54.230675 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.230633    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd" podStartSLOduration=9.381412751 podStartE2EDuration="29.230620082s" totalImagesPullingTime="19.849207331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:28:54.230274639 +0000 UTC m=+29.729635883" watchObservedRunningTime="2026-07-21 16:28:54.230620082 +0000 UTC m=+29.729981325"
Jul 21 16:28:54.937335 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.937307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:54.937645 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.937307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:54.937645 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:54.937419    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:54.937645 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:54.937307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:54.937645 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:54.937481    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:54.937645 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:54.937563    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:56.182138 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:56.182102    2580 generic.go:356] "Generic (PLEG): container finished" podID="b56ec3a5-eb74-4d28-b538-40c09c485e2b" containerID="dd3d1e32d3d02c3d2d7dde051a35e8e8c3a1f348f8b31b7f40021c8187273046" exitCode=0
Jul 21 16:28:56.182626 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:56.182163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t2fh" event={"ID":"b56ec3a5-eb74-4d28-b538-40c09c485e2b","Type":"ContainerDied","Data":"dd3d1e32d3d02c3d2d7dde051a35e8e8c3a1f348f8b31b7f40021c8187273046"}
Jul 21 16:28:56.934329 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:56.934293    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:56.934547 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:56.934297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:56.934547 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:56.934450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:56.934664 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:56.934570    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:56.934664 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:56.934425    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:56.934760 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:56.934667    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:28:58.188392 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:58.188356    2580 generic.go:356] "Generic (PLEG): container finished" podID="b56ec3a5-eb74-4d28-b538-40c09c485e2b" containerID="c6e907d6241ea4967d0d5e8e15a21ee43e169291281953e133f2042848792c98" exitCode=0
Jul 21 16:28:58.188924 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:58.188410    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t2fh" event={"ID":"b56ec3a5-eb74-4d28-b538-40c09c485e2b","Type":"ContainerDied","Data":"c6e907d6241ea4967d0d5e8e15a21ee43e169291281953e133f2042848792c98"}
Jul 21 16:28:58.686877 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:58.686849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:58.687034 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.687017    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:58.687094 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.687083    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs podName:54b53eb9-2e20-43cc-ad51-f15f75c19947 nodeName:}" failed. No retries permitted until 2026-07-21 16:29:30.687063906 +0000 UTC m=+66.186425131 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs") pod "network-metrics-daemon-k82tw" (UID: "54b53eb9-2e20-43cc-ad51-f15f75c19947") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 16:28:58.787757 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:58.787721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8ld\" (UniqueName: \"kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld\") pod \"network-check-target-qfgz6\" (UID: \"ec9487ce-c019-466f-9731-34e77376169f\") " pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:58.787757 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:58.787761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:58.787951 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.787840    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 16:28:58.787951 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.787850    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:58.787951 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.787859    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 16:28:58.787951 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.787869    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jq8ld for pod openshift-network-diagnostics/network-check-target-qfgz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:58.787951 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.787902    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret podName:77f6aedc-bb21-4183-bce6-420144e434b7 nodeName:}" failed. No retries permitted until 2026-07-21 16:29:30.78788971 +0000 UTC m=+66.287250939 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret") pod "global-pull-secret-syncer-h4rpx" (UID: "77f6aedc-bb21-4183-bce6-420144e434b7") : object "kube-system"/"original-pull-secret" not registered
Jul 21 16:28:58.787951 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.787917    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld podName:ec9487ce-c019-466f-9731-34e77376169f nodeName:}" failed. No retries permitted until 2026-07-21 16:29:30.787910883 +0000 UTC m=+66.287272105 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jq8ld" (UniqueName: "kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld") pod "network-check-target-qfgz6" (UID: "ec9487ce-c019-466f-9731-34e77376169f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 16:28:58.933698 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:58.933667    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:28:58.933698 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:58.933665    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:28:58.933920 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.933838    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:28:58.933920 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:28:58.933894    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:28:58.934030 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.934008    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:28:58.934106 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:28:58.934085    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:29:00.934015 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:00.933986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:00.934491 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:00.934033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:00.934491 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:00.934112    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:29:00.934491 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:00.934160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:00.934491 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:00.934249    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:29:00.934491 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:00.934464    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:29:02.934411 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:02.934384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:02.934411 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:02.934396    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:02.934773 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:02.934385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:02.934773 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:02.934497    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:29:02.934773 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:02.934588    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:29:02.934773 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:02.934666    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:29:03.204444 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:03.204370    2580 generic.go:356] "Generic (PLEG): container finished" podID="b56ec3a5-eb74-4d28-b538-40c09c485e2b" containerID="1d06d79cc6129956c4fa4a2a9633f093e810c5b1d3d6f9a871f1105d608da8e4" exitCode=0
Jul 21 16:29:03.204579 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:03.204449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t2fh" event={"ID":"b56ec3a5-eb74-4d28-b538-40c09c485e2b","Type":"ContainerDied","Data":"1d06d79cc6129956c4fa4a2a9633f093e810c5b1d3d6f9a871f1105d608da8e4"}
Jul 21 16:29:04.208711 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:04.208679    2580 generic.go:356] "Generic (PLEG): container finished" podID="b56ec3a5-eb74-4d28-b538-40c09c485e2b" containerID="616328e7f2c4037e98cdb1038684f56616559d41faeb40db7ddfd43f9c9cc2e5" exitCode=0
Jul 21 16:29:04.209081 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:04.208760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t2fh" event={"ID":"b56ec3a5-eb74-4d28-b538-40c09c485e2b","Type":"ContainerDied","Data":"616328e7f2c4037e98cdb1038684f56616559d41faeb40db7ddfd43f9c9cc2e5"}
Jul 21 16:29:04.935667 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:04.935636    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:04.935800 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:04.935718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:04.935800 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:04.935766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:04.935800 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:04.935760    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:29:04.935930 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:04.935845    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:29:04.935972 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:04.935941    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:29:05.213803 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:05.213722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t2fh" event={"ID":"b56ec3a5-eb74-4d28-b538-40c09c485e2b","Type":"ContainerStarted","Data":"153bcda7158660d5e8ce823a3308e192007c2afae367d92591d92919ee4db41f"}
Jul 21 16:29:05.235680 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:05.235639    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7t2fh" podStartSLOduration=9.192208446 podStartE2EDuration="40.235625949s" totalImagesPullingTime="31.043417503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:05.235548478 +0000 UTC m=+40.734909721" watchObservedRunningTime="2026-07-21 16:29:05.235625949 +0000 UTC m=+40.734987194"
Jul 21 16:29:06.934271 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:06.934242    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:06.934671 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:06.934334    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:29:06.934671 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:06.934243    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:06.934671 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:06.934394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:06.934671 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:06.934451    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:29:06.934671 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:06.934500    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:29:08.934286 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:08.934258    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:08.934286 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:08.934303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:08.934722 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:08.934327    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:08.934722 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:08.934426    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:29:08.934722 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:08.934470    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:29:08.934722 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:08.934601    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:29:10.817292 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:10.817118    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h4rpx"]
Jul 21 16:29:10.817946 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:10.817425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:10.817946 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:10.817553    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:29:10.820320 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:10.820296    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k82tw"]
Jul 21 16:29:10.820440 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:10.820405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:10.820556 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:10.820528    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:29:10.831920 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:10.831867    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qfgz6"]
Jul 21 16:29:10.832013 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:10.831945    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:10.832013 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:10.832006    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:29:12.934032 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:12.933997    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:12.934032 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:12.934012    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:12.934824 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:12.934098    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:12.934824 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:12.934109    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4rpx" podUID="77f6aedc-bb21-4183-bce6-420144e434b7"
Jul 21 16:29:12.934824 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:12.934181    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k82tw" podUID="54b53eb9-2e20-43cc-ad51-f15f75c19947"
Jul 21 16:29:12.934824 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:12.934246    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfgz6" podUID="ec9487ce-c019-466f-9731-34e77376169f"
Jul 21 16:29:14.881909 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.881833    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-158.ec2.internal" event="NodeReady"
Jul 21 16:29:14.882403 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.881950    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 16:29:14.931301 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.931273    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nwcd9"]
Jul 21 16:29:14.950874 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.950850    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwcd9"
Jul 21 16:29:14.953681 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.953663    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kw7ld"]
Jul 21 16:29:14.953783 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.953669    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 16:29:14.954213 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.954199    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c76lp\""
Jul 21 16:29:14.954302 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.954282    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 16:29:14.955783 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.955764    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 16:29:14.965267 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.965246    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nwcd9"]
Jul 21 16:29:14.965369 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.965277    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kw7ld"]
Jul 21 16:29:14.965369 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.965306    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2vgss"]
Jul 21 16:29:14.965369 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.965331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:14.965545 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.965331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:14.965545 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.965483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:14.965658 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.965644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:14.968899 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.968882    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 16:29:14.969079 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.969065    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 16:29:14.969136 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.969121    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 16:29:14.969183 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.969127    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 16:29:14.969234 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.969178    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 16:29:14.970010 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.969981    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dnpg8\""
Jul 21 16:29:14.970010 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.969999    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 16:29:14.970162 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.970010    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d444m\""
Jul 21 16:29:14.970162 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.970091    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 16:29:14.970162 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.970118    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xlxp4\""
Jul 21 16:29:14.970424 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.970407    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 16:29:14.981924 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.981908    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2vgss"]
Jul 21 16:29:14.982023 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.981976    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:14.984747 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.984729    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 16:29:14.985045 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.985028    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 16:29:14.985104 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:14.985028    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5fjqx\""
Jul 21 16:29:15.010585 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/590ad4b9-4708-4ced-9634-bb0791490700-cert\") pod \"ingress-canary-nwcd9\" (UID: \"590ad4b9-4708-4ced-9634-bb0791490700\") " pod="openshift-ingress-canary/ingress-canary-nwcd9"
Jul 21 16:29:15.010696 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d41000f-6150-492d-b613-d1b08ef6e5b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.010696 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-tmp-dir\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.010696 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pbtg\" (UniqueName: \"kubernetes.io/projected/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-kube-api-access-5pbtg\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.010696 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-metrics-tls\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.010889 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010711    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-config-volume\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.010889 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d41000f-6150-492d-b613-d1b08ef6e5b7-data-volume\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.010889 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kprf2\" (UniqueName: \"kubernetes.io/projected/590ad4b9-4708-4ced-9634-bb0791490700-kube-api-access-kprf2\") pod \"ingress-canary-nwcd9\" (UID: \"590ad4b9-4708-4ced-9634-bb0791490700\") " pod="openshift-ingress-canary/ingress-canary-nwcd9"
Jul 21 16:29:15.010889 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d41000f-6150-492d-b613-d1b08ef6e5b7-crio-socket\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.011061 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g846l\" (UniqueName: \"kubernetes.io/projected/8d41000f-6150-492d-b613-d1b08ef6e5b7-kube-api-access-g846l\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.011061 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.010949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d41000f-6150-492d-b613-d1b08ef6e5b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.111969 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.111945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d41000f-6150-492d-b613-d1b08ef6e5b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.112075 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.111973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/590ad4b9-4708-4ced-9634-bb0791490700-cert\") pod \"ingress-canary-nwcd9\" (UID: \"590ad4b9-4708-4ced-9634-bb0791490700\") " pod="openshift-ingress-canary/ingress-canary-nwcd9"
Jul 21 16:29:15.112075 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.111988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d41000f-6150-492d-b613-d1b08ef6e5b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.112075 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-tmp-dir\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.112075 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pbtg\" (UniqueName: \"kubernetes.io/projected/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-kube-api-access-5pbtg\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.112276 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-metrics-tls\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.112276 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-config-volume\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.112276 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d41000f-6150-492d-b613-d1b08ef6e5b7-data-volume\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.112276 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kprf2\" (UniqueName: \"kubernetes.io/projected/590ad4b9-4708-4ced-9634-bb0791490700-kube-api-access-kprf2\") pod \"ingress-canary-nwcd9\" (UID: \"590ad4b9-4708-4ced-9634-bb0791490700\") " pod="openshift-ingress-canary/ingress-canary-nwcd9"
Jul 21 16:29:15.112276 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d41000f-6150-492d-b613-d1b08ef6e5b7-crio-socket\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.112276 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112232    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g846l\" (UniqueName: \"kubernetes.io/projected/8d41000f-6150-492d-b613-d1b08ef6e5b7-kube-api-access-g846l\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.112543 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112296    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-tmp-dir\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.112595 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112575    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d41000f-6150-492d-b613-d1b08ef6e5b7-data-volume\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.112595 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d41000f-6150-492d-b613-d1b08ef6e5b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.112688 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112599    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d41000f-6150-492d-b613-d1b08ef6e5b7-crio-socket\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.112995 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.112969    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-config-volume\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.116307 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.116279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-metrics-tls\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.116469 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.116448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d41000f-6150-492d-b613-d1b08ef6e5b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.116555 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.116474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/590ad4b9-4708-4ced-9634-bb0791490700-cert\") pod \"ingress-canary-nwcd9\" (UID: \"590ad4b9-4708-4ced-9634-bb0791490700\") " pod="openshift-ingress-canary/ingress-canary-nwcd9"
Jul 21 16:29:15.120367 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.120347    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g846l\" (UniqueName: \"kubernetes.io/projected/8d41000f-6150-492d-b613-d1b08ef6e5b7-kube-api-access-g846l\") pod \"insights-runtime-extractor-kw7ld\" (UID: \"8d41000f-6150-492d-b613-d1b08ef6e5b7\") " pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.120367 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.120358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kprf2\" (UniqueName: \"kubernetes.io/projected/590ad4b9-4708-4ced-9634-bb0791490700-kube-api-access-kprf2\") pod \"ingress-canary-nwcd9\" (UID: \"590ad4b9-4708-4ced-9634-bb0791490700\") " pod="openshift-ingress-canary/ingress-canary-nwcd9"
Jul 21 16:29:15.120470 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.120441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pbtg\" (UniqueName: \"kubernetes.io/projected/5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5-kube-api-access-5pbtg\") pod \"dns-default-2vgss\" (UID: \"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5\") " pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.262814 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.262788    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwcd9"
Jul 21 16:29:15.274603 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.274582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kw7ld"
Jul 21 16:29:15.297349 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.297221    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:15.441705 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.441678    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nwcd9"]
Jul 21 16:29:15.445951 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.445926    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kw7ld"]
Jul 21 16:29:15.446050 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:15.446010    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590ad4b9_4708_4ced_9634_bb0791490700.slice/crio-9a805de911472994780d6199d81f4bb3854fa06a3851108aeefb96f7df4f28b8 WatchSource:0}: Error finding container 9a805de911472994780d6199d81f4bb3854fa06a3851108aeefb96f7df4f28b8: Status 404 returned error can't find the container with id 9a805de911472994780d6199d81f4bb3854fa06a3851108aeefb96f7df4f28b8
Jul 21 16:29:15.449605 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:15.449586    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d41000f_6150_492d_b613_d1b08ef6e5b7.slice/crio-6af38add6fb1165d4ff79ad45ccbd9faf5b10efd6ea8d9a38e6542500cda5c48 WatchSource:0}: Error finding container 6af38add6fb1165d4ff79ad45ccbd9faf5b10efd6ea8d9a38e6542500cda5c48: Status 404 returned error can't find the container with id 6af38add6fb1165d4ff79ad45ccbd9faf5b10efd6ea8d9a38e6542500cda5c48
Jul 21 16:29:15.459149 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:15.459130    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2vgss"]
Jul 21 16:29:15.461957 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:15.461937    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5edd3abf_bfb8_4ea8_aa78_6bbb5a4860f5.slice/crio-ba7d603cdfa543e55e42dadaaf82e553ed8ea46d899b6d5c2c9b63cbeb543559 WatchSource:0}: Error finding container ba7d603cdfa543e55e42dadaaf82e553ed8ea46d899b6d5c2c9b63cbeb543559: Status 404 returned error can't find the container with id ba7d603cdfa543e55e42dadaaf82e553ed8ea46d899b6d5c2c9b63cbeb543559
Jul 21 16:29:16.007684 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.007653    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"]
Jul 21 16:29:16.016445 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.016421    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"
Jul 21 16:29:16.019226 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.018730    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 21 16:29:16.019226 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.018824    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7q228\""
Jul 21 16:29:16.021315 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.021260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"]
Jul 21 16:29:16.125687 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.121855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96c47679-0ac0-4013-bf75-300d948e68ad-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ppqjx\" (UID: \"96c47679-0ac0-4013-bf75-300d948e68ad\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"
Jul 21 16:29:16.222825 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.222721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96c47679-0ac0-4013-bf75-300d948e68ad-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ppqjx\" (UID: \"96c47679-0ac0-4013-bf75-300d948e68ad\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"
Jul 21 16:29:16.222988 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:16.222908    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 21 16:29:16.222988 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:16.222986    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96c47679-0ac0-4013-bf75-300d948e68ad-tls-certificates podName:96c47679-0ac0-4013-bf75-300d948e68ad nodeName:}" failed. No retries permitted until 2026-07-21 16:29:16.722963999 +0000 UTC m=+52.222325236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/96c47679-0ac0-4013-bf75-300d948e68ad-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-ppqjx" (UID: "96c47679-0ac0-4013-bf75-300d948e68ad") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 21 16:29:16.240910 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.240797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kw7ld" event={"ID":"8d41000f-6150-492d-b613-d1b08ef6e5b7","Type":"ContainerStarted","Data":"6af38add6fb1165d4ff79ad45ccbd9faf5b10efd6ea8d9a38e6542500cda5c48"}
Jul 21 16:29:16.242138 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.242109    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nwcd9" event={"ID":"590ad4b9-4708-4ced-9634-bb0791490700","Type":"ContainerStarted","Data":"9a805de911472994780d6199d81f4bb3854fa06a3851108aeefb96f7df4f28b8"}
Jul 21 16:29:16.243708 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.243651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2vgss" event={"ID":"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5","Type":"ContainerStarted","Data":"ba7d603cdfa543e55e42dadaaf82e553ed8ea46d899b6d5c2c9b63cbeb543559"}
Jul 21 16:29:16.727949 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.727911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96c47679-0ac0-4013-bf75-300d948e68ad-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ppqjx\" (UID: \"96c47679-0ac0-4013-bf75-300d948e68ad\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"
Jul 21 16:29:16.740764 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.740730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96c47679-0ac0-4013-bf75-300d948e68ad-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ppqjx\" (UID: \"96c47679-0ac0-4013-bf75-300d948e68ad\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"
Jul 21 16:29:16.929904 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:16.929868    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"
Jul 21 16:29:17.187354 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.186413    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d4c8659b6-h7nc7"]
Jul 21 16:29:17.234176 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.234144    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d4c8659b6-h7nc7"]
Jul 21 16:29:17.234341 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.234265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.237760 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.237736    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bhspf\""
Jul 21 16:29:17.238014 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.237999    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 16:29:17.238198 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.238183    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 16:29:17.238480 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.238449    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 16:29:17.238676 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.238657    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 16:29:17.239620 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.238722    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 16:29:17.239620 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.238732    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 16:29:17.239620 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.238772    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 16:29:17.333617 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.333584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-oauth-config\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.333782 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.333623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-config\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.333782 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.333652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-service-ca\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.333782 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.333704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-oauth-serving-cert\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.333992 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.333771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-serving-cert\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.333992 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.333841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7lgp\" (UniqueName: \"kubernetes.io/projected/e5e6a628-7591-4ebe-995f-74d0e7a9a898-kube-api-access-l7lgp\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.434201 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.434168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-oauth-serving-cert\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.434358 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.434215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-serving-cert\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.434358 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.434260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7lgp\" (UniqueName: \"kubernetes.io/projected/e5e6a628-7591-4ebe-995f-74d0e7a9a898-kube-api-access-l7lgp\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.434358 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.434323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-oauth-config\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.434358 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.434344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-config\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.434572 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.434364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-service-ca\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.435067 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.435046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-service-ca\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.436137 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.435544    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-oauth-serving-cert\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.436137 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.436099    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-config\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.438503 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.438446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-oauth-config\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.439105 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.439084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-serving-cert\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.445073 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.445046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7lgp\" (UniqueName: \"kubernetes.io/projected/e5e6a628-7591-4ebe-995f-74d0e7a9a898-kube-api-access-l7lgp\") pod \"console-6d4c8659b6-h7nc7\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") " pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.546416 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.546238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:17.671463 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.671431    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"]
Jul 21 16:29:17.676234 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:17.676203    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c47679_0ac0_4013_bf75_300d948e68ad.slice/crio-5d4f9ff3c7b854dd3418601afe889947063dae47c33fe8edeb0c1f8ea952ca6c WatchSource:0}: Error finding container 5d4f9ff3c7b854dd3418601afe889947063dae47c33fe8edeb0c1f8ea952ca6c: Status 404 returned error can't find the container with id 5d4f9ff3c7b854dd3418601afe889947063dae47c33fe8edeb0c1f8ea952ca6c
Jul 21 16:29:17.723412 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:17.723351    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d4c8659b6-h7nc7"]
Jul 21 16:29:17.863819 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:17.863774    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e6a628_7591_4ebe_995f_74d0e7a9a898.slice/crio-b0cd0bd34fe4201d684b815e03019b8a1f5c8cb5e198f3643ae833a7a8ac3648 WatchSource:0}: Error finding container b0cd0bd34fe4201d684b815e03019b8a1f5c8cb5e198f3643ae833a7a8ac3648: Status 404 returned error can't find the container with id b0cd0bd34fe4201d684b815e03019b8a1f5c8cb5e198f3643ae833a7a8ac3648
Jul 21 16:29:18.250713 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:18.250642    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d4c8659b6-h7nc7" event={"ID":"e5e6a628-7591-4ebe-995f-74d0e7a9a898","Type":"ContainerStarted","Data":"b0cd0bd34fe4201d684b815e03019b8a1f5c8cb5e198f3643ae833a7a8ac3648"}
Jul 21 16:29:18.251923 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:18.251895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx" event={"ID":"96c47679-0ac0-4013-bf75-300d948e68ad","Type":"ContainerStarted","Data":"5d4f9ff3c7b854dd3418601afe889947063dae47c33fe8edeb0c1f8ea952ca6c"}
Jul 21 16:29:24.271549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.271292    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nwcd9" event={"ID":"590ad4b9-4708-4ced-9634-bb0791490700","Type":"ContainerStarted","Data":"d4735abf58faa0e14ceeda51750e4045046998d1df2725c0b2792db5d956b8fa"}
Jul 21 16:29:24.273533 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.273482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kw7ld" event={"ID":"8d41000f-6150-492d-b613-d1b08ef6e5b7","Type":"ContainerStarted","Data":"b95b143fd523c2ed587d06b35a6619a9e696c6af390a851bec4ed56252815770"}
Jul 21 16:29:24.276097 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.275984    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2vgss" event={"ID":"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5","Type":"ContainerStarted","Data":"ae82a1fbcc9d07e84929ee28deddb5edea955566ae178406d5c63a4ddc42b0cf"}
Jul 21 16:29:24.276097 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.276033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2vgss" event={"ID":"5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5","Type":"ContainerStarted","Data":"330d0d5f3aefa6df4095a7215fa1677b9daa7500cf5aeefbd62355538cd9dc90"}
Jul 21 16:29:24.276097 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.276063    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:24.278508 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.278484    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d4c8659b6-h7nc7" event={"ID":"e5e6a628-7591-4ebe-995f-74d0e7a9a898","Type":"ContainerStarted","Data":"04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e"}
Jul 21 16:29:24.280645 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.280594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx" event={"ID":"96c47679-0ac0-4013-bf75-300d948e68ad","Type":"ContainerStarted","Data":"39f1bde8dc7ccf611c26ca6349bd2908313d5089dd10eb459672d93970ad1731"}
Jul 21 16:29:24.280774 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.280753    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"
Jul 21 16:29:24.286947 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.286926    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx"
Jul 21 16:29:24.287543 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.287483    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nwcd9" podStartSLOduration=2.773463374 podStartE2EDuration="10.287468155s" totalImagesPullingTime="7.514004781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:24.286550779 +0000 UTC m=+59.785912027" watchObservedRunningTime="2026-07-21 16:29:24.287468155 +0000 UTC m=+59.786829400"
Jul 21 16:29:24.306799 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.306742    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d4c8659b6-h7nc7" podStartSLOduration=1.88879766 podStartE2EDuration="7.306722789s" totalImagesPullingTime="5.417925129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:24.305871574 +0000 UTC m=+59.805232818" watchObservedRunningTime="2026-07-21 16:29:24.306722789 +0000 UTC m=+59.806084024"
Jul 21 16:29:24.328971 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.326067    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2vgss" podStartSLOduration=3.867372381 podStartE2EDuration="10.326049746s" totalImagesPullingTime="6.458677365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:24.324330356 +0000 UTC m=+59.823691600" watchObservedRunningTime="2026-07-21 16:29:24.326049746 +0000 UTC m=+59.825410991"
Jul 21 16:29:24.343757 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.343709    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ppqjx" podStartSLOduration=3.743355058 podStartE2EDuration="9.34369053s" totalImagesPullingTime="5.600335472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:24.342976011 +0000 UTC m=+59.842337256" watchObservedRunningTime="2026-07-21 16:29:24.34369053 +0000 UTC m=+59.843051775"
Jul 21 16:29:24.960305 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.960275    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c4686467d-2pnlr"]
Jul 21 16:29:24.963322 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.963302    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:24.975321 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.975296    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c4686467d-2pnlr"]
Jul 21 16:29:24.975811 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:24.975791    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 21 16:29:25.080291 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.080268    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"]
Jul 21 16:29:25.086388 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.086361    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.089039 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.089017    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 21 16:29:25.089627 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.089174    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 21 16:29:25.089627 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.089248    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 16:29:25.089627 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.089304    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 16:29:25.089627 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.089438    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r7dv5\""
Jul 21 16:29:25.089627 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.089481    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 16:29:25.091611 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.091497    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"]
Jul 21 16:29:25.103642 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.103614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-serving-cert\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.103740 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.103641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-oauth-serving-cert\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.103803 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.103739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-oauth-config\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.103803 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.103769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-console-config\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.103912 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.103817    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-service-ca\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.103912 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.103850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-trusted-ca-bundle\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.103912 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.103877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4rrf\" (UniqueName: \"kubernetes.io/projected/66c36472-0a06-44a6-b503-c1da44e82a93-kube-api-access-p4rrf\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.204746 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.204714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.204909 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.204755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-serving-cert\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.204909 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.204783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4rrf\" (UniqueName: \"kubernetes.io/projected/66c36472-0a06-44a6-b503-c1da44e82a93-kube-api-access-p4rrf\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.205024 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.204967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-oauth-serving-cert\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.205077 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.205027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-578j2\" (UniqueName: \"kubernetes.io/projected/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-kube-api-access-578j2\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.205077 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.205059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-oauth-config\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.205180 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.205094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-trusted-ca-bundle\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.205180 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.205128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.205180 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.205148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-service-ca\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.205427 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.205191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.205427 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.205221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-console-config\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.205818 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.205749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-oauth-serving-cert\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.205818 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.205850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-service-ca\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.205818 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.205887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-console-config\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.206170 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.206030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-trusted-ca-bundle\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.208193 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.208173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-oauth-config\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.208362 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.208337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-serving-cert\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.212963 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.212908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4rrf\" (UniqueName: \"kubernetes.io/projected/66c36472-0a06-44a6-b503-c1da44e82a93-kube-api-access-p4rrf\") pod \"console-6c4686467d-2pnlr\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") " pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.274024 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.273994    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:25.306079 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.306042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.306189 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.306081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.306189 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.306158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-578j2\" (UniqueName: \"kubernetes.io/projected/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-kube-api-access-578j2\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.306358 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.306337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.307267 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.307243    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.308737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.308714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.309968 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.309949    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.316390 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.316371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-578j2\" (UniqueName: \"kubernetes.io/projected/bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb-kube-api-access-578j2\") pod \"prometheus-operator-5945b5cfd-pq4j8\" (UID: \"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:25.397288 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:25.397265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"
Jul 21 16:29:26.192897 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:26.192872    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gn5dd"
Jul 21 16:29:27.181686 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:27.181658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c4686467d-2pnlr"]
Jul 21 16:29:27.183932 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:27.183910    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c36472_0a06_44a6_b503_c1da44e82a93.slice/crio-541c6a55f07e54dd2ab143364a1546dffb8e7e3078a2cea0e943b7b187c5f28a WatchSource:0}: Error finding container 541c6a55f07e54dd2ab143364a1546dffb8e7e3078a2cea0e943b7b187c5f28a: Status 404 returned error can't find the container with id 541c6a55f07e54dd2ab143364a1546dffb8e7e3078a2cea0e943b7b187c5f28a
Jul 21 16:29:27.196527 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:27.196485    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8"]
Jul 21 16:29:27.198777 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:27.198749    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3ff912_f53c_4bc6_bae5_403ab5a6bcfb.slice/crio-b17e66bebe5da8090f7a61b1817f9045706cdb41a6234c4326a21fc63f94133e WatchSource:0}: Error finding container b17e66bebe5da8090f7a61b1817f9045706cdb41a6234c4326a21fc63f94133e: Status 404 returned error can't find the container with id b17e66bebe5da8090f7a61b1817f9045706cdb41a6234c4326a21fc63f94133e
Jul 21 16:29:27.297574 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:27.297540    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8" event={"ID":"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb","Type":"ContainerStarted","Data":"b17e66bebe5da8090f7a61b1817f9045706cdb41a6234c4326a21fc63f94133e"}
Jul 21 16:29:27.299221 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:27.299197    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4686467d-2pnlr" event={"ID":"66c36472-0a06-44a6-b503-c1da44e82a93","Type":"ContainerStarted","Data":"7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838"}
Jul 21 16:29:27.299322 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:27.299235    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4686467d-2pnlr" event={"ID":"66c36472-0a06-44a6-b503-c1da44e82a93","Type":"ContainerStarted","Data":"541c6a55f07e54dd2ab143364a1546dffb8e7e3078a2cea0e943b7b187c5f28a"}
Jul 21 16:29:27.317816 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:27.317721    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c4686467d-2pnlr" podStartSLOduration=3.317695449 podStartE2EDuration="3.317695449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:27.317037891 +0000 UTC m=+62.816399134" watchObservedRunningTime="2026-07-21 16:29:27.317695449 +0000 UTC m=+62.817056694"
Jul 21 16:29:27.547288 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:27.547265    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:27.547363 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:27.547315    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:29:27.548495 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:27.548474    2580 patch_prober.go:28] interesting pod/console-6d4c8659b6-h7nc7 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused" start-of-body=
Jul 21 16:29:27.548594 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:27.548531    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6d4c8659b6-h7nc7" podUID="e5e6a628-7591-4ebe-995f-74d0e7a9a898" containerName="console" probeResult="failure" output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused"
Jul 21 16:29:28.305548 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:28.305494    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kw7ld" event={"ID":"8d41000f-6150-492d-b613-d1b08ef6e5b7","Type":"ContainerStarted","Data":"215da87545e5f6f37edd2c73b360da34f70f764ad16f961e51e640d9a804b3e7"}
Jul 21 16:29:29.310375 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:29.310341    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8" event={"ID":"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb","Type":"ContainerStarted","Data":"ba19da317691207307af8b001eb1c7b5090344dd5bba75811991ab64c374fc36"}
Jul 21 16:29:29.310846 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:29.310400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8" event={"ID":"bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb","Type":"ContainerStarted","Data":"84d3280b19b4202cff0f830de6e86cd8ee60daa367f1fadaa65fcdcc7ff52c36"}
Jul 21 16:29:29.329258 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:29.329203    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pq4j8" podStartSLOduration=2.818481594 podStartE2EDuration="4.329180143s" totalImagesPullingTime="1.510698549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:29.326871033 +0000 UTC m=+64.826232302" watchObservedRunningTime="2026-07-21 16:29:29.329180143 +0000 UTC m=+64.828541390"
Jul 21 16:29:30.757587 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.757555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:30.760305 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.760279    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 16:29:30.770897 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.770874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b53eb9-2e20-43cc-ad51-f15f75c19947-metrics-certs\") pod \"network-metrics-daemon-k82tw\" (UID: \"54b53eb9-2e20-43cc-ad51-f15f75c19947\") " pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:30.858476 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.858435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8ld\" (UniqueName: \"kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld\") pod \"network-check-target-qfgz6\" (UID: \"ec9487ce-c019-466f-9731-34e77376169f\") " pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:30.858648 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.858560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:30.861229 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.861205    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 16:29:30.861381 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.861205    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 16:29:30.871440 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.871420    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 16:29:30.871850 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.871834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77f6aedc-bb21-4183-bce6-420144e434b7-original-pull-secret\") pod \"global-pull-secret-syncer-h4rpx\" (UID: \"77f6aedc-bb21-4183-bce6-420144e434b7\") " pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:30.882866 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.882681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq8ld\" (UniqueName: \"kubernetes.io/projected/ec9487ce-c019-466f-9731-34e77376169f-kube-api-access-jq8ld\") pod \"network-check-target-qfgz6\" (UID: \"ec9487ce-c019-466f-9731-34e77376169f\") " pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:30.887333 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.887300    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4rpx"
Jul 21 16:29:30.894743 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.894725    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xlxp4\""
Jul 21 16:29:30.902966 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:30.902943    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k82tw"
Jul 21 16:29:31.186098 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.185856    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dnpg8\""
Jul 21 16:29:31.193780 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.193753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:31.291696 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.291669    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h4rpx"]
Jul 21 16:29:31.294684 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:31.294662    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f6aedc_bb21_4183_bce6_420144e434b7.slice/crio-994ae73ae6db802838e4d8c77083e222f07401c76b687063944689993da14f2b WatchSource:0}: Error finding container 994ae73ae6db802838e4d8c77083e222f07401c76b687063944689993da14f2b: Status 404 returned error can't find the container with id 994ae73ae6db802838e4d8c77083e222f07401c76b687063944689993da14f2b
Jul 21 16:29:31.309648 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.309554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k82tw"]
Jul 21 16:29:31.313153 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:31.313126    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b53eb9_2e20_43cc_ad51_f15f75c19947.slice/crio-d56d9ee6610df9b0ab01d068fbc8530720e14240a9730e74780bcc9b5d29c23e WatchSource:0}: Error finding container d56d9ee6610df9b0ab01d068fbc8530720e14240a9730e74780bcc9b5d29c23e: Status 404 returned error can't find the container with id d56d9ee6610df9b0ab01d068fbc8530720e14240a9730e74780bcc9b5d29c23e
Jul 21 16:29:31.324331 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.324268    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k82tw" event={"ID":"54b53eb9-2e20-43cc-ad51-f15f75c19947","Type":"ContainerStarted","Data":"d56d9ee6610df9b0ab01d068fbc8530720e14240a9730e74780bcc9b5d29c23e"}
Jul 21 16:29:31.327140 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.326672    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kw7ld" event={"ID":"8d41000f-6150-492d-b613-d1b08ef6e5b7","Type":"ContainerStarted","Data":"b6c5f95d31aec96336f436fcda1eaea0c733cb0baee60500e2511e28c42e4e60"}
Jul 21 16:29:31.328020 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.327996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h4rpx" event={"ID":"77f6aedc-bb21-4183-bce6-420144e434b7","Type":"ContainerStarted","Data":"994ae73ae6db802838e4d8c77083e222f07401c76b687063944689993da14f2b"}
Jul 21 16:29:31.345758 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.345716    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kw7ld" podStartSLOduration=1.871192104 podStartE2EDuration="17.345695582s" totalImagesPullingTime="15.474503478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:29:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:31.345374994 +0000 UTC m=+66.844736258" watchObservedRunningTime="2026-07-21 16:29:31.345695582 +0000 UTC m=+66.845056826"
Jul 21 16:29:31.350790 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.350770    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qfgz6"]
Jul 21 16:29:31.353314 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:31.353293    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec9487ce_c019_466f_9731_34e77376169f.slice/crio-aabf91ce3878011050089de8fd946b1f81dbcfe2be04d9da49fd4c8c22a4aef7 WatchSource:0}: Error finding container aabf91ce3878011050089de8fd946b1f81dbcfe2be04d9da49fd4c8c22a4aef7: Status 404 returned error can't find the container with id aabf91ce3878011050089de8fd946b1f81dbcfe2be04d9da49fd4c8c22a4aef7
Jul 21 16:29:31.464944 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.464922    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"]
Jul 21 16:29:31.469203 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.469188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.471729 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.471713    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 21 16:29:31.471810 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.471731    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9mckk\""
Jul 21 16:29:31.471995 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.471978    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 21 16:29:31.481483 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.481465    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"]
Jul 21 16:29:31.485074 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.485058    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f7mf5"]
Jul 21 16:29:31.488079 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.488062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.490466 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.490444    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 16:29:31.490581 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.490504    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 16:29:31.490581 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.490527    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 16:29:31.490581 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.490553    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-77mcm\""
Jul 21 16:29:31.562304 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-tls\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.562378 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d72670a4-5a3a-43b6-9b1f-2ec241b90647-sys\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.562378 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbbaca30-b505-4768-80e6-34c932966aa7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.562449 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-textfile\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.562449 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.562530 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562449    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d72670a4-5a3a-43b6-9b1f-2ec241b90647-root\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.562530 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-wtmp\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.562530 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwcq6\" (UniqueName: \"kubernetes.io/projected/d72670a4-5a3a-43b6-9b1f-2ec241b90647-kube-api-access-jwcq6\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.562619 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbbaca30-b505-4768-80e6-34c932966aa7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.562619 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhvkn\" (UniqueName: \"kubernetes.io/projected/bbbaca30-b505-4768-80e6-34c932966aa7-kube-api-access-jhvkn\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.562619 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d72670a4-5a3a-43b6-9b1f-2ec241b90647-metrics-client-ca\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.562619 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-accelerators-collector-config\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.562738 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.562642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbbaca30-b505-4768-80e6-34c932966aa7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.663567 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-textfile\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.663672 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.663672 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d72670a4-5a3a-43b6-9b1f-2ec241b90647-root\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.663672 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-wtmp\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.663838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwcq6\" (UniqueName: \"kubernetes.io/projected/d72670a4-5a3a-43b6-9b1f-2ec241b90647-kube-api-access-jwcq6\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.663838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbbaca30-b505-4768-80e6-34c932966aa7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.663838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhvkn\" (UniqueName: \"kubernetes.io/projected/bbbaca30-b505-4768-80e6-34c932966aa7-kube-api-access-jhvkn\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.663838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d72670a4-5a3a-43b6-9b1f-2ec241b90647-root\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.663838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d72670a4-5a3a-43b6-9b1f-2ec241b90647-metrics-client-ca\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.663838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-accelerators-collector-config\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.664125 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-wtmp\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.664125 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbbaca30-b505-4768-80e6-34c932966aa7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.664125 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-tls\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.664125 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d72670a4-5a3a-43b6-9b1f-2ec241b90647-sys\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.664125 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.663936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbbaca30-b505-4768-80e6-34c932966aa7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.664125 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:31.664018    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 21 16:29:31.664125 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.664058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d72670a4-5a3a-43b6-9b1f-2ec241b90647-sys\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.664125 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:31.664078    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-tls podName:d72670a4-5a3a-43b6-9b1f-2ec241b90647 nodeName:}" failed. No retries permitted until 2026-07-21 16:29:32.164057326 +0000 UTC m=+67.663418569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-tls") pod "node-exporter-f7mf5" (UID: "d72670a4-5a3a-43b6-9b1f-2ec241b90647") : secret "node-exporter-tls" not found
Jul 21 16:29:31.664680 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:31.664163    2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 21 16:29:31.664680 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:31.664214    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbbaca30-b505-4768-80e6-34c932966aa7-openshift-state-metrics-tls podName:bbbaca30-b505-4768-80e6-34c932966aa7 nodeName:}" failed. No retries permitted until 2026-07-21 16:29:32.164195152 +0000 UTC m=+67.663556392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/bbbaca30-b505-4768-80e6-34c932966aa7-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-bcxcd" (UID: "bbbaca30-b505-4768-80e6-34c932966aa7") : secret "openshift-state-metrics-tls" not found
Jul 21 16:29:31.664680 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.664419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d72670a4-5a3a-43b6-9b1f-2ec241b90647-metrics-client-ca\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.664680 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.664492    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-textfile\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.664680 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.664663    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-accelerators-collector-config\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.664862 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.664703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbbaca30-b505-4768-80e6-34c932966aa7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.666103 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.666084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.666268 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.666249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbbaca30-b505-4768-80e6-34c932966aa7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:31.675784 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.675738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwcq6\" (UniqueName: \"kubernetes.io/projected/d72670a4-5a3a-43b6-9b1f-2ec241b90647-kube-api-access-jwcq6\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:31.676655 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:31.676637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhvkn\" (UniqueName: \"kubernetes.io/projected/bbbaca30-b505-4768-80e6-34c932966aa7-kube-api-access-jhvkn\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:32.168036 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:32.167990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbbaca30-b505-4768-80e6-34c932966aa7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:32.168575 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:32.168047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-tls\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:32.168575 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:32.168203    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 21 16:29:32.168575 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:32.168271    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-tls podName:d72670a4-5a3a-43b6-9b1f-2ec241b90647 nodeName:}" failed. No retries permitted until 2026-07-21 16:29:33.168251955 +0000 UTC m=+68.667613199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-tls") pod "node-exporter-f7mf5" (UID: "d72670a4-5a3a-43b6-9b1f-2ec241b90647") : secret "node-exporter-tls" not found
Jul 21 16:29:32.175335 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:32.175304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbbaca30-b505-4768-80e6-34c932966aa7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-bcxcd\" (UID: \"bbbaca30-b505-4768-80e6-34c932966aa7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:32.335653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:32.335578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qfgz6" event={"ID":"ec9487ce-c019-466f-9731-34e77376169f","Type":"ContainerStarted","Data":"aabf91ce3878011050089de8fd946b1f81dbcfe2be04d9da49fd4c8c22a4aef7"}
Jul 21 16:29:32.378468 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:32.378427    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"
Jul 21 16:29:32.707302 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:32.707208    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd"]
Jul 21 16:29:32.708649 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:32.708609    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbbaca30_b505_4768_80e6_34c932966aa7.slice/crio-cc7c3cb589fd1c4094e342221bf2c38b07cf68d328d0e42fa76516c94b928e3c WatchSource:0}: Error finding container cc7c3cb589fd1c4094e342221bf2c38b07cf68d328d0e42fa76516c94b928e3c: Status 404 returned error can't find the container with id cc7c3cb589fd1c4094e342221bf2c38b07cf68d328d0e42fa76516c94b928e3c
Jul 21 16:29:33.177635 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:33.177598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-tls\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:33.180457 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:33.180429    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d72670a4-5a3a-43b6-9b1f-2ec241b90647-node-exporter-tls\") pod \"node-exporter-f7mf5\" (UID: \"d72670a4-5a3a-43b6-9b1f-2ec241b90647\") " pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:33.296549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:33.296499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f7mf5"
Jul 21 16:29:33.309035 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:33.308997    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd72670a4_5a3a_43b6_9b1f_2ec241b90647.slice/crio-b901bb918b72bc8b0ef1c0fbe63f355ef6f73c187e6f3543e789cfe4d6eb6cf6 WatchSource:0}: Error finding container b901bb918b72bc8b0ef1c0fbe63f355ef6f73c187e6f3543e789cfe4d6eb6cf6: Status 404 returned error can't find the container with id b901bb918b72bc8b0ef1c0fbe63f355ef6f73c187e6f3543e789cfe4d6eb6cf6
Jul 21 16:29:33.344026 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:33.343983    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k82tw" event={"ID":"54b53eb9-2e20-43cc-ad51-f15f75c19947","Type":"ContainerStarted","Data":"eeead1d33a57e1156f2e8bd47d5cd739f73d0c9af5501c5530f3284224333218"}
Jul 21 16:29:33.344148 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:33.344091    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k82tw" event={"ID":"54b53eb9-2e20-43cc-ad51-f15f75c19947","Type":"ContainerStarted","Data":"6b57166dd9cc3a13c43f296abebc4526814e8de893b355ec0e5d4167fad4ca14"}
Jul 21 16:29:33.345987 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:33.345959    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7mf5" event={"ID":"d72670a4-5a3a-43b6-9b1f-2ec241b90647","Type":"ContainerStarted","Data":"b901bb918b72bc8b0ef1c0fbe63f355ef6f73c187e6f3543e789cfe4d6eb6cf6"}
Jul 21 16:29:33.347970 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:33.347944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd" event={"ID":"bbbaca30-b505-4768-80e6-34c932966aa7","Type":"ContainerStarted","Data":"01f4e95f1510c912fdc2b36598dbe9a8db358c79a3216013868d6325e26c0526"}
Jul 21 16:29:33.348069 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:33.347992    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd" event={"ID":"bbbaca30-b505-4768-80e6-34c932966aa7","Type":"ContainerStarted","Data":"1cdfa68a4cde5ece6dc38b4cbede23c23e401b04686181a0c947089533121238"}
Jul 21 16:29:33.348069 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:33.348010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd" event={"ID":"bbbaca30-b505-4768-80e6-34c932966aa7","Type":"ContainerStarted","Data":"cc7c3cb589fd1c4094e342221bf2c38b07cf68d328d0e42fa76516c94b928e3c"}
Jul 21 16:29:33.361608 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:33.361570    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k82tw" podStartSLOduration=67.096469343 podStartE2EDuration="1m8.36155392s" totalImagesPullingTime="1.265084577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:33.360725468 +0000 UTC m=+68.860086713" watchObservedRunningTime="2026-07-21 16:29:33.36155392 +0000 UTC m=+68.860915166"
Jul 21 16:29:34.286343 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:34.286307    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2vgss"
Jul 21 16:29:35.274399 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.274365    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:35.274593 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.274427    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:35.281183 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.281157    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:35.362576 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.362552    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:29:35.414295 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.414264    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d4c8659b6-h7nc7"]
Jul 21 16:29:35.757044 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.757007    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-d5d9cf589-7hdrl"]
Jul 21 16:29:35.760092 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.760070    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:35.764259 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.764013    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 21 16:29:35.764259 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.764056    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 21 16:29:35.764259 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.764068    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fxc9m\""
Jul 21 16:29:35.764259 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.764112    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 21 16:29:35.764259 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.764157    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 21 16:29:35.764589 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.764421    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-560pisrs912g3\""
Jul 21 16:29:35.768601 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.768556    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d5d9cf589-7hdrl"]
Jul 21 16:29:35.904841 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.904797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ddd8a70b-6d1c-465a-a4db-f224964f7416-metrics-server-audit-profiles\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:35.904841 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.904841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddd8a70b-6d1c-465a-a4db-f224964f7416-client-ca-bundle\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:35.905053 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.904882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ddd8a70b-6d1c-465a-a4db-f224964f7416-secret-metrics-server-tls\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:35.905053 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.904947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ddd8a70b-6d1c-465a-a4db-f224964f7416-secret-metrics-server-client-certs\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:35.905053 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.904986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs8j9\" (UniqueName: \"kubernetes.io/projected/ddd8a70b-6d1c-465a-a4db-f224964f7416-kube-api-access-xs8j9\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:35.905053 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.905016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd8a70b-6d1c-465a-a4db-f224964f7416-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:35.905053 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:35.905037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ddd8a70b-6d1c-465a-a4db-f224964f7416-audit-log\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.005918 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.005879    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ddd8a70b-6d1c-465a-a4db-f224964f7416-metrics-server-audit-profiles\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.005918 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.005917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddd8a70b-6d1c-465a-a4db-f224964f7416-client-ca-bundle\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.006131 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.005948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ddd8a70b-6d1c-465a-a4db-f224964f7416-secret-metrics-server-tls\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.006131 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.005992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ddd8a70b-6d1c-465a-a4db-f224964f7416-secret-metrics-server-client-certs\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.006131 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.006018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs8j9\" (UniqueName: \"kubernetes.io/projected/ddd8a70b-6d1c-465a-a4db-f224964f7416-kube-api-access-xs8j9\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.006131 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.006053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd8a70b-6d1c-465a-a4db-f224964f7416-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.006131 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.006117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ddd8a70b-6d1c-465a-a4db-f224964f7416-audit-log\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.006708 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.006434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ddd8a70b-6d1c-465a-a4db-f224964f7416-audit-log\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.006991 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.006965    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd8a70b-6d1c-465a-a4db-f224964f7416-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.007162 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.007112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ddd8a70b-6d1c-465a-a4db-f224964f7416-metrics-server-audit-profiles\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.008865 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.008842    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ddd8a70b-6d1c-465a-a4db-f224964f7416-secret-metrics-server-tls\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.008997 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.008978    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddd8a70b-6d1c-465a-a4db-f224964f7416-client-ca-bundle\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.009093 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.009072    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ddd8a70b-6d1c-465a-a4db-f224964f7416-secret-metrics-server-client-certs\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.015388 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.015344    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs8j9\" (UniqueName: \"kubernetes.io/projected/ddd8a70b-6d1c-465a-a4db-f224964f7416-kube-api-access-xs8j9\") pod \"metrics-server-d5d9cf589-7hdrl\" (UID: \"ddd8a70b-6d1c-465a-a4db-f224964f7416\") " pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.071258 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.071228    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:36.230837 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.230811    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"]
Jul 21 16:29:36.233350 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.233328    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:36.235536 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.235496    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-snx5t\""
Jul 21 16:29:36.235698 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.235568    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 21 16:29:36.241967 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.241927    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"]
Jul 21 16:29:36.307873 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.307789    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtnww\" (UniqueName: \"kubernetes.io/projected/b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e-kube-api-access-xtnww\") pod \"monitoring-plugin-7b7b7bc8bf-46hwk\" (UID: \"b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:36.308029 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.307878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-46hwk\" (UID: \"b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:36.408582 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.408547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtnww\" (UniqueName: \"kubernetes.io/projected/b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e-kube-api-access-xtnww\") pod \"monitoring-plugin-7b7b7bc8bf-46hwk\" (UID: \"b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:36.409031 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.408625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-46hwk\" (UID: \"b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:36.409031 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:36.408731    2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 21 16:29:36.409031 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:29:36.408795    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e-monitoring-plugin-cert podName:b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e nodeName:}" failed. No retries permitted until 2026-07-21 16:29:36.908774035 +0000 UTC m=+72.408135263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-46hwk" (UID: "b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e") : secret "monitoring-plugin-cert" not found
Jul 21 16:29:36.417435 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.417414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtnww\" (UniqueName: \"kubernetes.io/projected/b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e-kube-api-access-xtnww\") pod \"monitoring-plugin-7b7b7bc8bf-46hwk\" (UID: \"b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:36.914196 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.914155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-46hwk\" (UID: \"b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:36.916969 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:36.916943    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-46hwk\" (UID: \"b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:37.122307 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.122225    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d5d9cf589-7hdrl"]
Jul 21 16:29:37.126105 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:37.126014    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd8a70b_6d1c_465a_a4db_f224964f7416.slice/crio-32e8f072e7181ab04b09db2fec12925c52b08d5f649a1674fdb2f331070b06a6 WatchSource:0}: Error finding container 32e8f072e7181ab04b09db2fec12925c52b08d5f649a1674fdb2f331070b06a6: Status 404 returned error can't find the container with id 32e8f072e7181ab04b09db2fec12925c52b08d5f649a1674fdb2f331070b06a6
Jul 21 16:29:37.148874 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.147089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:37.282355 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.282285    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"]
Jul 21 16:29:37.285219 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:37.285197    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4b94a8a_ea7d_4b7b_bc45_1bc371c6ef2e.slice/crio-0e97e0bbf687ddc06c6dbb3c86930dd77c624c0ef9312384f59ef9f62a9c2ca3 WatchSource:0}: Error finding container 0e97e0bbf687ddc06c6dbb3c86930dd77c624c0ef9312384f59ef9f62a9c2ca3: Status 404 returned error can't find the container with id 0e97e0bbf687ddc06c6dbb3c86930dd77c624c0ef9312384f59ef9f62a9c2ca3
Jul 21 16:29:37.363966 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.363935    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qfgz6" event={"ID":"ec9487ce-c019-466f-9731-34e77376169f","Type":"ContainerStarted","Data":"9afc8c4de4b97d6f7f583d7b36b9f88cc15fcf121d8be85953d6d0d90019af4d"}
Jul 21 16:29:37.364095 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.364025    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:29:37.365143 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.365116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl" event={"ID":"ddd8a70b-6d1c-465a-a4db-f224964f7416","Type":"ContainerStarted","Data":"32e8f072e7181ab04b09db2fec12925c52b08d5f649a1674fdb2f331070b06a6"}
Jul 21 16:29:37.366945 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.366920    2580 generic.go:356] "Generic (PLEG): container finished" podID="d72670a4-5a3a-43b6-9b1f-2ec241b90647" containerID="50b6f5de007eea715d264107f78d199fbc240e318e2fee800d07c88e5e79abc7" exitCode=0
Jul 21 16:29:37.367068 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.367016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7mf5" event={"ID":"d72670a4-5a3a-43b6-9b1f-2ec241b90647","Type":"ContainerDied","Data":"50b6f5de007eea715d264107f78d199fbc240e318e2fee800d07c88e5e79abc7"}
Jul 21 16:29:37.368651 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.368633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h4rpx" event={"ID":"77f6aedc-bb21-4183-bce6-420144e434b7","Type":"ContainerStarted","Data":"a901520905ce5b796c96a06e663e4b1966e1104bfcd4705e3e6ef7306da0cea6"}
Jul 21 16:29:37.370838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.370819    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd" event={"ID":"bbbaca30-b505-4768-80e6-34c932966aa7","Type":"ContainerStarted","Data":"c88f99a5a00cc6121e3f59b0185496906639b1ca4b24ac9b397b3285e883f989"}
Jul 21 16:29:37.372122 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.372106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk" event={"ID":"b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e","Type":"ContainerStarted","Data":"0e97e0bbf687ddc06c6dbb3c86930dd77c624c0ef9312384f59ef9f62a9c2ca3"}
Jul 21 16:29:37.379088 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.379050    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qfgz6" podStartSLOduration=66.784098916 podStartE2EDuration="1m12.379037615s" totalImagesPullingTime="5.594938699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:37.378401636 +0000 UTC m=+72.877762880" watchObservedRunningTime="2026-07-21 16:29:37.379037615 +0000 UTC m=+72.878398836"
Jul 21 16:29:37.397400 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.397362    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bcxcd" podStartSLOduration=2.311272922 podStartE2EDuration="6.397350527s" totalImagesPullingTime="4.086077605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:37.396474035 +0000 UTC m=+72.895835297" watchObservedRunningTime="2026-07-21 16:29:37.397350527 +0000 UTC m=+72.896711771"
Jul 21 16:29:37.433780 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.433740    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h4rpx" podStartSLOduration=66.780311604 podStartE2EDuration="1m12.433727665s" totalImagesPullingTime="5.653416061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:37.432332405 +0000 UTC m=+72.931693649" watchObservedRunningTime="2026-07-21 16:29:37.433727665 +0000 UTC m=+72.933088906"
Jul 21 16:29:37.772246 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.772213    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 16:29:37.776507 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.776482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.779092    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9ea30dcq5tg7r\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.779422    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qxr25\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.779684    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.780250    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.780279    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.780431    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.780591    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.780661    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.780685    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.780753    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.780765    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.780803    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 21 16:29:37.780965 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.780878    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 21 16:29:37.781572 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.781498    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 21 16:29:37.785128 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.784477    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 21 16:29:37.786070 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.785883    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 21 16:29:37.793056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.793025    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 16:29:37.924572 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.924741 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924583    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.924741 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.924741 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.924741 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.924920 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.924920 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.924920 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n66c4\" (UniqueName: \"kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-kube-api-access-n66c4\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.924920 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.924920 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.924920 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924903    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.925117 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.925117 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.925117 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924980    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-config\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.925117 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.924994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.925117 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.925075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.925316 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.925123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.925316 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.925146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.925316 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.925164    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:37.925316 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:37.925181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.026646 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.026646 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.026838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-config\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.026838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.026838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.026838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.026838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.026838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.026838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.027195 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.027195 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.027195 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.027195 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.027195 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.026961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.027446 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.027360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.028032 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.028005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.028279 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.028258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.028386 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.028298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.028386 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.028340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n66c4\" (UniqueName: \"kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-kube-api-access-n66c4\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.028386 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.028371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.028575 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.028397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.028575 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.028430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.029969 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.029627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.029969 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.029694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.032151 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.031372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.032151 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.031775    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-config\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.033348 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.033033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.033348 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.033077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.035577 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.035551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.036127 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.036077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.036127 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.036080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.036246 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.036186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.036246 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.036234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.036676 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.036656    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.036737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.036714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.036781 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.036741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.037258 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.037238    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.037738 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.037706    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.038585 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.038562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.040888 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.040844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n66c4\" (UniqueName: \"kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-kube-api-access-n66c4\") pod \"prometheus-k8s-0\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.089083 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.089033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:38.094306 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.094280    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c5d8cbbdd-7xm5h"]
Jul 21 16:29:38.098184 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.098164    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.108341 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.108282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c5d8cbbdd-7xm5h"]
Jul 21 16:29:38.238159 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.236862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-oauth-config\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.238159 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.236933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fcmc\" (UniqueName: \"kubernetes.io/projected/73085c34-3ff3-449b-8adf-8bf70b4577cb-kube-api-access-7fcmc\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.238159 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.237015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-trusted-ca-bundle\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.238159 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.237044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-service-ca\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.238159 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.237175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-config\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.238159 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.237222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-serving-cert\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.238159 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.237247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-oauth-serving-cert\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.338686 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.338590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-trusted-ca-bundle\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.338686 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.338648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-service-ca\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.338883 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.338692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-config\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.338883 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.338722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-serving-cert\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.338883 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.338747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-oauth-serving-cert\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.338883 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.338808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-oauth-config\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.338883 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.338844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fcmc\" (UniqueName: \"kubernetes.io/projected/73085c34-3ff3-449b-8adf-8bf70b4577cb-kube-api-access-7fcmc\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.339553 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.339497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-config\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.339672 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.339591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-service-ca\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.339672 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.339591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-oauth-serving-cert\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.339672 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.339661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-trusted-ca-bundle\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.342046 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.342026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-oauth-config\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.342248 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.342229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-serving-cert\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.351359 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.351335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fcmc\" (UniqueName: \"kubernetes.io/projected/73085c34-3ff3-449b-8adf-8bf70b4577cb-kube-api-access-7fcmc\") pod \"console-6c5d8cbbdd-7xm5h\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") " pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:38.379562 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.379416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7mf5" event={"ID":"d72670a4-5a3a-43b6-9b1f-2ec241b90647","Type":"ContainerStarted","Data":"81ef8bf8523c1ed4f46eec2847f424f1244a124f1a538c89ab0eee4736a39641"}
Jul 21 16:29:38.379562 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.379528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7mf5" event={"ID":"d72670a4-5a3a-43b6-9b1f-2ec241b90647","Type":"ContainerStarted","Data":"d294e8d1b45f258aaa0ef88e60408bba71efaa4de2b9483b725ec4f43303031b"}
Jul 21 16:29:38.398667 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.398624    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f7mf5" podStartSLOduration=3.603894888 podStartE2EDuration="7.39861103s" totalImagesPullingTime="3.707544238s" totalInitContainerRuntime="87.171904ms" isStatefulPod=true podCreationTimestamp="2026-07-21 16:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:38.397890485 +0000 UTC m=+73.897251748" watchObservedRunningTime="2026-07-21 16:29:38.39861103 +0000 UTC m=+73.897972265"
Jul 21 16:29:38.411899 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:38.411878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:39.173156 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.173131    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 16:29:39.174710 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:39.174685    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a030a99_a173_4060_835a_6b344c11ca82.slice/crio-a5ed721c967f9ef0fcc3511586ade95207207bac0122c408d449ecb7743d7a22 WatchSource:0}: Error finding container a5ed721c967f9ef0fcc3511586ade95207207bac0122c408d449ecb7743d7a22: Status 404 returned error can't find the container with id a5ed721c967f9ef0fcc3511586ade95207207bac0122c408d449ecb7743d7a22
Jul 21 16:29:39.183904 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.183881    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c5d8cbbdd-7xm5h"]
Jul 21 16:29:39.186943 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:39.186923    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73085c34_3ff3_449b_8adf_8bf70b4577cb.slice/crio-cb3d6539805f40c9080bfa0a58f15583e4c98afab285b9077bdbb2b02c533d5a WatchSource:0}: Error finding container cb3d6539805f40c9080bfa0a58f15583e4c98afab285b9077bdbb2b02c533d5a: Status 404 returned error can't find the container with id cb3d6539805f40c9080bfa0a58f15583e4c98afab285b9077bdbb2b02c533d5a
Jul 21 16:29:39.385079 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.384998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl" event={"ID":"ddd8a70b-6d1c-465a-a4db-f224964f7416","Type":"ContainerStarted","Data":"6f3ee627f378e36f1e17284e41ba63fc071febf1f2da1ca6d1fcf03b7d71c8d3"}
Jul 21 16:29:39.386683 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.386656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c5d8cbbdd-7xm5h" event={"ID":"73085c34-3ff3-449b-8adf-8bf70b4577cb","Type":"ContainerStarted","Data":"d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1"}
Jul 21 16:29:39.386797 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.386703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c5d8cbbdd-7xm5h" event={"ID":"73085c34-3ff3-449b-8adf-8bf70b4577cb","Type":"ContainerStarted","Data":"cb3d6539805f40c9080bfa0a58f15583e4c98afab285b9077bdbb2b02c533d5a"}
Jul 21 16:29:39.387947 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.387922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerStarted","Data":"a5ed721c967f9ef0fcc3511586ade95207207bac0122c408d449ecb7743d7a22"}
Jul 21 16:29:39.389353 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.389327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk" event={"ID":"b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e","Type":"ContainerStarted","Data":"9501ebc15abe71e1116a710f54fa6b5eea3d11aef4511e5b8d3aa77ef0842a14"}
Jul 21 16:29:39.389651 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.389636    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:39.394592 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.394574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk"
Jul 21 16:29:39.412047 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.412011    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl" podStartSLOduration=2.533926534 podStartE2EDuration="4.411992303s" totalImagesPullingTime="1.878065769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:39.411723934 +0000 UTC m=+74.911085179" watchObservedRunningTime="2026-07-21 16:29:39.411992303 +0000 UTC m=+74.911353547"
Jul 21 16:29:39.438933 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.438897    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c5d8cbbdd-7xm5h" podStartSLOduration=1.438883887 podStartE2EDuration="1.438883887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:39.438071631 +0000 UTC m=+74.937432876" watchObservedRunningTime="2026-07-21 16:29:39.438883887 +0000 UTC m=+74.938245131"
Jul 21 16:29:39.468175 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:39.468138    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-46hwk" podStartSLOduration=1.748444871 podStartE2EDuration="3.468123378s" totalImagesPullingTime="1.719678507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:39.467283013 +0000 UTC m=+74.966644258" watchObservedRunningTime="2026-07-21 16:29:39.468123378 +0000 UTC m=+74.967484622"
Jul 21 16:29:40.395173 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:40.395143    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a030a99-a173-4060-835a-6b344c11ca82" containerID="dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634" exitCode=0
Jul 21 16:29:40.395555 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:40.395237    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerDied","Data":"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"}
Jul 21 16:29:41.670685 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.670654    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c5d8cbbdd-7xm5h"]
Jul 21 16:29:41.719194 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.719149    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dfbc9b98b-rqfc8"]
Jul 21 16:29:41.722675 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.722653    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.735997 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.735972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dfbc9b98b-rqfc8"]
Jul 21 16:29:41.873832 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.873802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-service-ca\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.873990 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.873852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kp7h\" (UniqueName: \"kubernetes.io/projected/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-kube-api-access-5kp7h\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.873990 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.873889    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-oauth-config\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.873990 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.873916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-config\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.873990 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.873973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-trusted-ca-bundle\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.874172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.874003    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-serving-cert\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.874172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.874034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-oauth-serving-cert\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.974786 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.974711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-service-ca\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.974786 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.974756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kp7h\" (UniqueName: \"kubernetes.io/projected/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-kube-api-access-5kp7h\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.974786 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.974781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-oauth-config\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.975048 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.974925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-config\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.975048 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.974997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-trusted-ca-bundle\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.975048 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.975037    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-serving-cert\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.975194 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.975066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-oauth-serving-cert\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.975546 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.975494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-service-ca\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.975670 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.975624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-config\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.975771 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.975752    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-oauth-serving-cert\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.975860 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.975836    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-trusted-ca-bundle\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.978117 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.978092    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-serving-cert\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.978786 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.978761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-oauth-config\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:41.984956 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:41.984935    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kp7h\" (UniqueName: \"kubernetes.io/projected/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-kube-api-access-5kp7h\") pod \"console-dfbc9b98b-rqfc8\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") " pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:42.037686 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:42.037660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:42.185944 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:42.185919    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dfbc9b98b-rqfc8"]
Jul 21 16:29:42.188634 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:29:42.188603    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef1e3d4_0161_4006_9a25_5cf98ce2ae3a.slice/crio-8e89dcba0e3cbf0106e4774e03f7af949e05f22a109bca95f3c3b787514d189b WatchSource:0}: Error finding container 8e89dcba0e3cbf0106e4774e03f7af949e05f22a109bca95f3c3b787514d189b: Status 404 returned error can't find the container with id 8e89dcba0e3cbf0106e4774e03f7af949e05f22a109bca95f3c3b787514d189b
Jul 21 16:29:42.405064 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:42.405028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfbc9b98b-rqfc8" event={"ID":"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a","Type":"ContainerStarted","Data":"4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442"}
Jul 21 16:29:42.405236 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:42.405090    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfbc9b98b-rqfc8" event={"ID":"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a","Type":"ContainerStarted","Data":"8e89dcba0e3cbf0106e4774e03f7af949e05f22a109bca95f3c3b787514d189b"}
Jul 21 16:29:42.423988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:42.423933    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dfbc9b98b-rqfc8" podStartSLOduration=1.423914498 podStartE2EDuration="1.423914498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:42.422482032 +0000 UTC m=+77.921843274" watchObservedRunningTime="2026-07-21 16:29:42.423914498 +0000 UTC m=+77.923275741"
Jul 21 16:29:44.417395 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:44.417358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerStarted","Data":"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"}
Jul 21 16:29:44.417899 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:44.417445    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerStarted","Data":"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"}
Jul 21 16:29:47.433256 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:47.433221    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerStarted","Data":"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"}
Jul 21 16:29:47.433726 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:47.433283    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerStarted","Data":"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"}
Jul 21 16:29:47.433726 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:47.433310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerStarted","Data":"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"}
Jul 21 16:29:47.433726 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:47.433334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerStarted","Data":"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"}
Jul 21 16:29:47.467055 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:47.467005    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.360274671 podStartE2EDuration="10.46698879s" totalImagesPullingTime="7.085717041s" totalInitContainerRuntime="20.997078ms" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:47.464275163 +0000 UTC m=+82.963636407" watchObservedRunningTime="2026-07-21 16:29:47.46698879 +0000 UTC m=+82.966350035"
Jul 21 16:29:48.089350 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:48.089295    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:29:48.412734 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:48.412708    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:29:52.038227 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:52.038189    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:52.038644 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:52.038246    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:52.043112 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:52.043088    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:52.461901 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:52.461874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:29:52.513376 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:52.513351    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c4686467d-2pnlr"]
Jul 21 16:29:56.071652 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:56.071622    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:29:56.072202 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:29:56.071677    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:30:00.437620 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:00.437556    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d4c8659b6-h7nc7" podUID="e5e6a628-7591-4ebe-995f-74d0e7a9a898" containerName="console" containerID="cri-o://04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e" gracePeriod=15
Jul 21 16:30:01.232922 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.232893    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:30:01.355571 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.355477    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-serving-cert\") pod \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") "
Jul 21 16:30:01.355571 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.355527    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-config\") pod \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") "
Jul 21 16:30:01.355571 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.355562    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-oauth-config\") pod \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") "
Jul 21 16:30:01.355571 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.355577    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7lgp\" (UniqueName: \"kubernetes.io/projected/e5e6a628-7591-4ebe-995f-74d0e7a9a898-kube-api-access-l7lgp\") pod \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") "
Jul 21 16:30:01.355887 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.355616    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-oauth-serving-cert\") pod \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") "
Jul 21 16:30:01.355887 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.355655    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-service-ca\") pod \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\" (UID: \"e5e6a628-7591-4ebe-995f-74d0e7a9a898\") "
Jul 21 16:30:01.356034 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.356012    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-config" (OuterVolumeSpecName: "console-config") pod "e5e6a628-7591-4ebe-995f-74d0e7a9a898" (UID: "e5e6a628-7591-4ebe-995f-74d0e7a9a898"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:01.356134 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.356042    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e5e6a628-7591-4ebe-995f-74d0e7a9a898" (UID: "e5e6a628-7591-4ebe-995f-74d0e7a9a898"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:01.356292 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.356238    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-service-ca" (OuterVolumeSpecName: "service-ca") pod "e5e6a628-7591-4ebe-995f-74d0e7a9a898" (UID: "e5e6a628-7591-4ebe-995f-74d0e7a9a898"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:01.358188 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.358162    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5e6a628-7591-4ebe-995f-74d0e7a9a898-kube-api-access-l7lgp" (OuterVolumeSpecName: "kube-api-access-l7lgp") pod "e5e6a628-7591-4ebe-995f-74d0e7a9a898" (UID: "e5e6a628-7591-4ebe-995f-74d0e7a9a898"). InnerVolumeSpecName "kube-api-access-l7lgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:30:01.358188 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.358170    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e5e6a628-7591-4ebe-995f-74d0e7a9a898" (UID: "e5e6a628-7591-4ebe-995f-74d0e7a9a898"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:01.358316 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.358239    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e5e6a628-7591-4ebe-995f-74d0e7a9a898" (UID: "e5e6a628-7591-4ebe-995f-74d0e7a9a898"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:01.456816 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.456789    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:01.456816 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.456815    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-serving-cert\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:01.457166 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.456825    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:01.457166 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.456836    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5e6a628-7591-4ebe-995f-74d0e7a9a898-console-oauth-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:01.457166 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.456844    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7lgp\" (UniqueName: \"kubernetes.io/projected/e5e6a628-7591-4ebe-995f-74d0e7a9a898-kube-api-access-l7lgp\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:01.457166 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.456851    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5e6a628-7591-4ebe-995f-74d0e7a9a898-oauth-serving-cert\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:01.498908 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.498879    2580 generic.go:356] "Generic (PLEG): container finished" podID="e5e6a628-7591-4ebe-995f-74d0e7a9a898" containerID="04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e" exitCode=0
Jul 21 16:30:01.499054 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.498930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d4c8659b6-h7nc7" event={"ID":"e5e6a628-7591-4ebe-995f-74d0e7a9a898","Type":"ContainerDied","Data":"04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e"}
Jul 21 16:30:01.499054 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.498935    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d4c8659b6-h7nc7"
Jul 21 16:30:01.499054 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.498992    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d4c8659b6-h7nc7" event={"ID":"e5e6a628-7591-4ebe-995f-74d0e7a9a898","Type":"ContainerDied","Data":"b0cd0bd34fe4201d684b815e03019b8a1f5c8cb5e198f3643ae833a7a8ac3648"}
Jul 21 16:30:01.499054 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.499020    2580 scope.go:117] "RemoveContainer" containerID="04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e"
Jul 21 16:30:01.508560 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.508540    2580 scope.go:117] "RemoveContainer" containerID="04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e"
Jul 21 16:30:01.508834 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:30:01.508815    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e\": container with ID starting with 04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e not found: ID does not exist" containerID="04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e"
Jul 21 16:30:01.508887 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.508845    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e"} err="failed to get container status \"04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e\": rpc error: code = NotFound desc = could not find container \"04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e\": container with ID starting with 04ede4eecc9eebe3f9ed01cc6235dd5d13d91536f22a2177fb558eb27cb18a9e not found: ID does not exist"
Jul 21 16:30:01.520176 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.520152    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d4c8659b6-h7nc7"]
Jul 21 16:30:01.523881 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:01.523858    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d4c8659b6-h7nc7"]
Jul 21 16:30:02.938458 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:02.938429    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5e6a628-7591-4ebe-995f-74d0e7a9a898" path="/var/lib/kubelet/pods/e5e6a628-7591-4ebe-995f-74d0e7a9a898/volumes"
Jul 21 16:30:06.696829 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:06.696759    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c5d8cbbdd-7xm5h" podUID="73085c34-3ff3-449b-8adf-8bf70b4577cb" containerName="console" containerID="cri-o://d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1" gracePeriod=15
Jul 21 16:30:06.775911 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:06.775887    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nwcd9_590ad4b9-4708-4ced-9634-bb0791490700/serve-healthcheck-canary/0.log"
Jul 21 16:30:06.932468 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:06.932445    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:30:07.103295 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103209    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-trusted-ca-bundle\") pod \"73085c34-3ff3-449b-8adf-8bf70b4577cb\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") "
Jul 21 16:30:07.103295 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103253    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-oauth-config\") pod \"73085c34-3ff3-449b-8adf-8bf70b4577cb\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") "
Jul 21 16:30:07.103295 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103297    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-oauth-serving-cert\") pod \"73085c34-3ff3-449b-8adf-8bf70b4577cb\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") "
Jul 21 16:30:07.103608 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103484    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-service-ca\") pod \"73085c34-3ff3-449b-8adf-8bf70b4577cb\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") "
Jul 21 16:30:07.103608 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103575    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-serving-cert\") pod \"73085c34-3ff3-449b-8adf-8bf70b4577cb\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") "
Jul 21 16:30:07.103721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103621    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fcmc\" (UniqueName: \"kubernetes.io/projected/73085c34-3ff3-449b-8adf-8bf70b4577cb-kube-api-access-7fcmc\") pod \"73085c34-3ff3-449b-8adf-8bf70b4577cb\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") "
Jul 21 16:30:07.103721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103634    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "73085c34-3ff3-449b-8adf-8bf70b4577cb" (UID: "73085c34-3ff3-449b-8adf-8bf70b4577cb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:07.103721 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103644    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-config\") pod \"73085c34-3ff3-449b-8adf-8bf70b4577cb\" (UID: \"73085c34-3ff3-449b-8adf-8bf70b4577cb\") "
Jul 21 16:30:07.103880 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103760    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "73085c34-3ff3-449b-8adf-8bf70b4577cb" (UID: "73085c34-3ff3-449b-8adf-8bf70b4577cb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:07.103947 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103926    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-trusted-ca-bundle\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:07.104004 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103958    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-oauth-serving-cert\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:07.104004 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103923    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-service-ca" (OuterVolumeSpecName: "service-ca") pod "73085c34-3ff3-449b-8adf-8bf70b4577cb" (UID: "73085c34-3ff3-449b-8adf-8bf70b4577cb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:07.104004 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.103952    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-config" (OuterVolumeSpecName: "console-config") pod "73085c34-3ff3-449b-8adf-8bf70b4577cb" (UID: "73085c34-3ff3-449b-8adf-8bf70b4577cb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:07.105769 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.105746    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "73085c34-3ff3-449b-8adf-8bf70b4577cb" (UID: "73085c34-3ff3-449b-8adf-8bf70b4577cb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:07.105888 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.105770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73085c34-3ff3-449b-8adf-8bf70b4577cb-kube-api-access-7fcmc" (OuterVolumeSpecName: "kube-api-access-7fcmc") pod "73085c34-3ff3-449b-8adf-8bf70b4577cb" (UID: "73085c34-3ff3-449b-8adf-8bf70b4577cb"). InnerVolumeSpecName "kube-api-access-7fcmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:30:07.105888 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.105770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "73085c34-3ff3-449b-8adf-8bf70b4577cb" (UID: "73085c34-3ff3-449b-8adf-8bf70b4577cb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:07.205325 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.205292    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:07.205325 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.205320    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-serving-cert\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:07.205325 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.205331    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fcmc\" (UniqueName: \"kubernetes.io/projected/73085c34-3ff3-449b-8adf-8bf70b4577cb-kube-api-access-7fcmc\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:07.205555 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.205341    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:07.205555 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.205351    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73085c34-3ff3-449b-8adf-8bf70b4577cb-console-oauth-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:07.524665 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.524629    2580 generic.go:356] "Generic (PLEG): container finished" podID="73085c34-3ff3-449b-8adf-8bf70b4577cb" containerID="d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1" exitCode=0
Jul 21 16:30:07.524821 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.524681    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c5d8cbbdd-7xm5h"
Jul 21 16:30:07.524821 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.524727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c5d8cbbdd-7xm5h" event={"ID":"73085c34-3ff3-449b-8adf-8bf70b4577cb","Type":"ContainerDied","Data":"d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1"}
Jul 21 16:30:07.524821 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.524784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c5d8cbbdd-7xm5h" event={"ID":"73085c34-3ff3-449b-8adf-8bf70b4577cb","Type":"ContainerDied","Data":"cb3d6539805f40c9080bfa0a58f15583e4c98afab285b9077bdbb2b02c533d5a"}
Jul 21 16:30:07.524821 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.524805    2580 scope.go:117] "RemoveContainer" containerID="d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1"
Jul 21 16:30:07.533571 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.533554    2580 scope.go:117] "RemoveContainer" containerID="d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1"
Jul 21 16:30:07.533858 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:30:07.533840    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1\": container with ID starting with d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1 not found: ID does not exist" containerID="d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1"
Jul 21 16:30:07.533926 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.533871    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1"} err="failed to get container status \"d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1\": rpc error: code = NotFound desc = could not find container \"d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1\": container with ID starting with d8d9be0c3203cdd3b89acde7c367398eae0d3c4cb99238354d199811ea0cc2d1 not found: ID does not exist"
Jul 21 16:30:07.545189 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.545168    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c5d8cbbdd-7xm5h"]
Jul 21 16:30:07.549749 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:07.549727    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c5d8cbbdd-7xm5h"]
Jul 21 16:30:08.382425 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:08.382390    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qfgz6"
Jul 21 16:30:08.938233 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:08.938199    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73085c34-3ff3-449b-8adf-8bf70b4577cb" path="/var/lib/kubelet/pods/73085c34-3ff3-449b-8adf-8bf70b4577cb/volumes"
Jul 21 16:30:16.077303 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:16.077272    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:30:16.081130 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:16.081106    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-d5d9cf589-7hdrl"
Jul 21 16:30:17.533979 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.533910    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c4686467d-2pnlr" podUID="66c36472-0a06-44a6-b503-c1da44e82a93" containerName="console" containerID="cri-o://7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838" gracePeriod=15
Jul 21 16:30:17.788188 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.788112    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:30:17.885948 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.885915    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4rrf\" (UniqueName: \"kubernetes.io/projected/66c36472-0a06-44a6-b503-c1da44e82a93-kube-api-access-p4rrf\") pod \"66c36472-0a06-44a6-b503-c1da44e82a93\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") "
Jul 21 16:30:17.886139 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.885980    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-console-config\") pod \"66c36472-0a06-44a6-b503-c1da44e82a93\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") "
Jul 21 16:30:17.886139 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.886010    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-oauth-serving-cert\") pod \"66c36472-0a06-44a6-b503-c1da44e82a93\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") "
Jul 21 16:30:17.886139 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.886040    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-serving-cert\") pod \"66c36472-0a06-44a6-b503-c1da44e82a93\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") "
Jul 21 16:30:17.886139 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.886085    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-trusted-ca-bundle\") pod \"66c36472-0a06-44a6-b503-c1da44e82a93\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") "
Jul 21 16:30:17.886139 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.886126    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-oauth-config\") pod \"66c36472-0a06-44a6-b503-c1da44e82a93\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") "
Jul 21 16:30:17.886396 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.886148    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-service-ca\") pod \"66c36472-0a06-44a6-b503-c1da44e82a93\" (UID: \"66c36472-0a06-44a6-b503-c1da44e82a93\") "
Jul 21 16:30:17.886452 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.886385    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "66c36472-0a06-44a6-b503-c1da44e82a93" (UID: "66c36472-0a06-44a6-b503-c1da44e82a93"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:17.886452 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.886393    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-console-config" (OuterVolumeSpecName: "console-config") pod "66c36472-0a06-44a6-b503-c1da44e82a93" (UID: "66c36472-0a06-44a6-b503-c1da44e82a93"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:17.886727 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.886697    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-service-ca" (OuterVolumeSpecName: "service-ca") pod "66c36472-0a06-44a6-b503-c1da44e82a93" (UID: "66c36472-0a06-44a6-b503-c1da44e82a93"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:17.886846 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.886818    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "66c36472-0a06-44a6-b503-c1da44e82a93" (UID: "66c36472-0a06-44a6-b503-c1da44e82a93"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:17.888675 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.888650    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "66c36472-0a06-44a6-b503-c1da44e82a93" (UID: "66c36472-0a06-44a6-b503-c1da44e82a93"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:17.888801 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.888776    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "66c36472-0a06-44a6-b503-c1da44e82a93" (UID: "66c36472-0a06-44a6-b503-c1da44e82a93"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:17.888921 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.888900    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66c36472-0a06-44a6-b503-c1da44e82a93-kube-api-access-p4rrf" (OuterVolumeSpecName: "kube-api-access-p4rrf") pod "66c36472-0a06-44a6-b503-c1da44e82a93" (UID: "66c36472-0a06-44a6-b503-c1da44e82a93"). InnerVolumeSpecName "kube-api-access-p4rrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:30:17.987737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.987696    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4rrf\" (UniqueName: \"kubernetes.io/projected/66c36472-0a06-44a6-b503-c1da44e82a93-kube-api-access-p4rrf\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:17.987737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.987736    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-console-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:17.987950 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.987752    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-oauth-serving-cert\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:17.987950 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.987766    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-serving-cert\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:17.987950 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.987779    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-trusted-ca-bundle\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:17.987950 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.987792    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66c36472-0a06-44a6-b503-c1da44e82a93-console-oauth-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:17.987950 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:17.987804    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66c36472-0a06-44a6-b503-c1da44e82a93-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:18.570206 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:18.570171    2580 generic.go:356] "Generic (PLEG): container finished" podID="66c36472-0a06-44a6-b503-c1da44e82a93" containerID="7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838" exitCode=0
Jul 21 16:30:18.570619 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:18.570229    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4686467d-2pnlr"
Jul 21 16:30:18.570619 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:18.570257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4686467d-2pnlr" event={"ID":"66c36472-0a06-44a6-b503-c1da44e82a93","Type":"ContainerDied","Data":"7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838"}
Jul 21 16:30:18.570619 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:18.570311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4686467d-2pnlr" event={"ID":"66c36472-0a06-44a6-b503-c1da44e82a93","Type":"ContainerDied","Data":"541c6a55f07e54dd2ab143364a1546dffb8e7e3078a2cea0e943b7b187c5f28a"}
Jul 21 16:30:18.570619 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:18.570335    2580 scope.go:117] "RemoveContainer" containerID="7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838"
Jul 21 16:30:18.581666 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:18.581643    2580 scope.go:117] "RemoveContainer" containerID="7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838"
Jul 21 16:30:18.582327 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:30:18.582307    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838\": container with ID starting with 7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838 not found: ID does not exist" containerID="7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838"
Jul 21 16:30:18.582410 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:18.582336    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838"} err="failed to get container status \"7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838\": rpc error: code = NotFound desc = could not find container \"7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838\": container with ID starting with 7a56f80b4978190c9394839af84abd49c88c5b967d14171fb6aeaab821776838 not found: ID does not exist"
Jul 21 16:30:18.595613 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:18.595585    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c4686467d-2pnlr"]
Jul 21 16:30:18.599364 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:18.599340    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c4686467d-2pnlr"]
Jul 21 16:30:18.939056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:18.939023    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66c36472-0a06-44a6-b503-c1da44e82a93" path="/var/lib/kubelet/pods/66c36472-0a06-44a6-b503-c1da44e82a93/volumes"
Jul 21 16:30:38.089280 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:38.089247    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:38.108396 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:38.108372    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:38.663856 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:38.663831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:53.029308 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.029277    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-889b45655-24ggj"]
Jul 21 16:30:53.029826 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.029554    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73085c34-3ff3-449b-8adf-8bf70b4577cb" containerName="console"
Jul 21 16:30:53.029826 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.029566    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="73085c34-3ff3-449b-8adf-8bf70b4577cb" containerName="console"
Jul 21 16:30:53.029826 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.029576    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5e6a628-7591-4ebe-995f-74d0e7a9a898" containerName="console"
Jul 21 16:30:53.029826 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.029581    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5e6a628-7591-4ebe-995f-74d0e7a9a898" containerName="console"
Jul 21 16:30:53.029826 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.029588    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c36472-0a06-44a6-b503-c1da44e82a93" containerName="console"
Jul 21 16:30:53.029826 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.029594    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c36472-0a06-44a6-b503-c1da44e82a93" containerName="console"
Jul 21 16:30:53.029826 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.029649    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="66c36472-0a06-44a6-b503-c1da44e82a93" containerName="console"
Jul 21 16:30:53.029826 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.029658    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5e6a628-7591-4ebe-995f-74d0e7a9a898" containerName="console"
Jul 21 16:30:53.029826 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.029664    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="73085c34-3ff3-449b-8adf-8bf70b4577cb" containerName="console"
Jul 21 16:30:53.032671 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.032654    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.042808 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.042783    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-889b45655-24ggj"]
Jul 21 16:30:53.138453 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.138422    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-console-config\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.138610 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.138486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-oauth-config\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.138610 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.138533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-trusted-ca-bundle\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.138610 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.138558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-oauth-serving-cert\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.138610 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.138586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-serving-cert\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.138610 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.138605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-service-ca\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.138768 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.138690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54vtp\" (UniqueName: \"kubernetes.io/projected/83bb5813-e437-4b56-b967-c192cfa1f035-kube-api-access-54vtp\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.239581 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.239543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-serving-cert\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.239581 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.239585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-service-ca\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.239875 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.239758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54vtp\" (UniqueName: \"kubernetes.io/projected/83bb5813-e437-4b56-b967-c192cfa1f035-kube-api-access-54vtp\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.239875 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.239813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-console-config\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.239875 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.239867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-oauth-config\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.240035 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.239890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-trusted-ca-bundle\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.240035 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.239912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-oauth-serving-cert\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.240317 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.240296    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-service-ca\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.240686 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.240666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-oauth-serving-cert\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.240751 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.240700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-console-config\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.240751 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.240742    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-trusted-ca-bundle\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.242051 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.242035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-serving-cert\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.242307 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.242290    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-oauth-config\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.248197 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.248174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54vtp\" (UniqueName: \"kubernetes.io/projected/83bb5813-e437-4b56-b967-c192cfa1f035-kube-api-access-54vtp\") pod \"console-889b45655-24ggj\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") " pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.342654 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.342586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:30:53.474528 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.474492    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-889b45655-24ggj"]
Jul 21 16:30:53.477060 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:30:53.477023    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83bb5813_e437_4b56_b967_c192cfa1f035.slice/crio-b208311df4ffacde07b185d17e1c1bb2557fb811c28bc1d1f2774c8a905adc9a WatchSource:0}: Error finding container b208311df4ffacde07b185d17e1c1bb2557fb811c28bc1d1f2774c8a905adc9a: Status 404 returned error can't find the container with id b208311df4ffacde07b185d17e1c1bb2557fb811c28bc1d1f2774c8a905adc9a
Jul 21 16:30:53.707672 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.707635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-889b45655-24ggj" event={"ID":"83bb5813-e437-4b56-b967-c192cfa1f035","Type":"ContainerStarted","Data":"b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d"}
Jul 21 16:30:53.707859 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.707697    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-889b45655-24ggj" event={"ID":"83bb5813-e437-4b56-b967-c192cfa1f035","Type":"ContainerStarted","Data":"b208311df4ffacde07b185d17e1c1bb2557fb811c28bc1d1f2774c8a905adc9a"}
Jul 21 16:30:53.725398 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:53.725355    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-889b45655-24ggj" podStartSLOduration=0.725339809 podStartE2EDuration="725.339809ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:30:53.723719348 +0000 UTC m=+149.223080596" watchObservedRunningTime="2026-07-21 16:30:53.725339809 +0000 UTC m=+149.224701052"
Jul 21 16:30:56.178234 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.178199    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 16:30:56.178670 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.178498    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="prometheus" containerID="cri-o://fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf" gracePeriod=600
Jul 21 16:30:56.178670 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.178560    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy" containerID="cri-o://6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8" gracePeriod=600
Jul 21 16:30:56.178670 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.178588    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab" gracePeriod=600
Jul 21 16:30:56.178849 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.178614    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy-web" containerID="cri-o://fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57" gracePeriod=600
Jul 21 16:30:56.178849 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.178587    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="thanos-sidecar" containerID="cri-o://987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471" gracePeriod=600
Jul 21 16:30:56.178849 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.178590    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="config-reloader" containerID="cri-o://35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd" gracePeriod=600
Jul 21 16:30:56.432293 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.432229    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.567214 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567186    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-thanos-prometheus-http-client-file\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567214 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567219    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-metrics-client-ca\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567454 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567241    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-tls\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567454 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567256    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-2\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567454 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567291    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-trusted-ca-bundle\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567642 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-db\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567642 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567537    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-1\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567642 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567566    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-config\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567642 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567621    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-tls-assets\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567642 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567624    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:56.567878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567646    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-kube-rbac-proxy\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567648    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:56.567878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567680    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-kubelet-serving-ca-bundle\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567692    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:56.567878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567717    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-serving-certs-ca-bundle\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-metrics-client-certs\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567775    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-0\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567847    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-web-config\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.567878 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567876    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.568325 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567913    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-config-out\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.568325 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567965    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n66c4\" (UniqueName: \"kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-kube-api-access-n66c4\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.568325 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.567991    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-grpc-tls\") pod \"4a030a99-a173-4060-835a-6b344c11ca82\" (UID: \"4a030a99-a173-4060-835a-6b344c11ca82\") "
Jul 21 16:30:56.568325 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.568223    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-metrics-client-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.568325 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.568240    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.568325 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.568255    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.569854 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.568971    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:56.569854 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.569357    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:56.569854 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.569547    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:56.570862 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.570832    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 16:30:56.571074 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.571048    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:56.571207 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.571085    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:56.571362 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.571345    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-config" (OuterVolumeSpecName: "config") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:56.571558 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.571456    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:56.571802 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.571772    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:56.571915 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.571887    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:30:56.572687 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.572654    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:56.572796 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.572770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:30:56.572872 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.572761    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-config-out" (OuterVolumeSpecName: "config-out") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 16:30:56.573051 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.573027    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:56.573261 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.573245    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-kube-api-access-n66c4" (OuterVolumeSpecName: "kube-api-access-n66c4") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "kube-api-access-n66c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:30:56.573505 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.573485    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:56.582328 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.582305    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-web-config" (OuterVolumeSpecName: "web-config") pod "4a030a99-a173-4060-835a-6b344c11ca82" (UID: "4a030a99-a173-4060-835a-6b344c11ca82"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:30:56.669506 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669478    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669506 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669502    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669538    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-db\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669554    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669568    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669578    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-tls-assets\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669585    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-kube-rbac-proxy\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669594    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669602    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669610    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-metrics-client-certs\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669620    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669629    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a030a99-a173-4060-835a-6b344c11ca82-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669637    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-web-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669645    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.669653 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669653    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a030a99-a173-4060-835a-6b344c11ca82-config-out\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.670027 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669662    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n66c4\" (UniqueName: \"kubernetes.io/projected/4a030a99-a173-4060-835a-6b344c11ca82-kube-api-access-n66c4\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.670027 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.669670    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a030a99-a173-4060-835a-6b344c11ca82-secret-grpc-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:30:56.722270 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722214    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a030a99-a173-4060-835a-6b344c11ca82" containerID="3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab" exitCode=0
Jul 21 16:30:56.722270 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722235    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a030a99-a173-4060-835a-6b344c11ca82" containerID="6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8" exitCode=0
Jul 21 16:30:56.722270 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722242    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a030a99-a173-4060-835a-6b344c11ca82" containerID="fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57" exitCode=0
Jul 21 16:30:56.722270 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722248    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a030a99-a173-4060-835a-6b344c11ca82" containerID="987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471" exitCode=0
Jul 21 16:30:56.722270 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722253    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a030a99-a173-4060-835a-6b344c11ca82" containerID="35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd" exitCode=0
Jul 21 16:30:56.722270 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722258    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a030a99-a173-4060-835a-6b344c11ca82" containerID="fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf" exitCode=0
Jul 21 16:30:56.722270 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722258    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerDied","Data":"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"}
Jul 21 16:30:56.722537 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722293    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.722537 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerDied","Data":"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"}
Jul 21 16:30:56.722537 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerDied","Data":"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"}
Jul 21 16:30:56.722537 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerDied","Data":"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"}
Jul 21 16:30:56.722537 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722416    2580 scope.go:117] "RemoveContainer" containerID="3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"
Jul 21 16:30:56.722537 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerDied","Data":"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"}
Jul 21 16:30:56.722537 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerDied","Data":"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"}
Jul 21 16:30:56.722537 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.722496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a030a99-a173-4060-835a-6b344c11ca82","Type":"ContainerDied","Data":"a5ed721c967f9ef0fcc3511586ade95207207bac0122c408d449ecb7743d7a22"}
Jul 21 16:30:56.730221 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.730146    2580 scope.go:117] "RemoveContainer" containerID="6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"
Jul 21 16:30:56.736909 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.736895    2580 scope.go:117] "RemoveContainer" containerID="fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"
Jul 21 16:30:56.743261 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.743243    2580 scope.go:117] "RemoveContainer" containerID="987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"
Jul 21 16:30:56.746392 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.746373    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 16:30:56.750323 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.750306    2580 scope.go:117] "RemoveContainer" containerID="35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"
Jul 21 16:30:56.751923 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.751905    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 16:30:56.756723 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.756706    2580 scope.go:117] "RemoveContainer" containerID="fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"
Jul 21 16:30:56.763351 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.763334    2580 scope.go:117] "RemoveContainer" containerID="dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"
Jul 21 16:30:56.769585 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.769566    2580 scope.go:117] "RemoveContainer" containerID="3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"
Jul 21 16:30:56.769844 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:30:56.769818    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": container with ID starting with 3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab not found: ID does not exist" containerID="3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"
Jul 21 16:30:56.769924 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.769852    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"} err="failed to get container status \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": rpc error: code = NotFound desc = could not find container \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": container with ID starting with 3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab not found: ID does not exist"
Jul 21 16:30:56.769924 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.769876    2580 scope.go:117] "RemoveContainer" containerID="6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"
Jul 21 16:30:56.770124 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:30:56.770107    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": container with ID starting with 6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8 not found: ID does not exist" containerID="6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"
Jul 21 16:30:56.770159 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.770128    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"} err="failed to get container status \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": rpc error: code = NotFound desc = could not find container \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": container with ID starting with 6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8 not found: ID does not exist"
Jul 21 16:30:56.770159 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.770140    2580 scope.go:117] "RemoveContainer" containerID="fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"
Jul 21 16:30:56.770358 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:30:56.770343    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": container with ID starting with fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57 not found: ID does not exist" containerID="fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"
Jul 21 16:30:56.770395 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.770367    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"} err="failed to get container status \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": rpc error: code = NotFound desc = could not find container \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": container with ID starting with fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57 not found: ID does not exist"
Jul 21 16:30:56.770395 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.770385    2580 scope.go:117] "RemoveContainer" containerID="987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"
Jul 21 16:30:56.770659 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:30:56.770641    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": container with ID starting with 987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471 not found: ID does not exist" containerID="987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"
Jul 21 16:30:56.770710 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.770663    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"} err="failed to get container status \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": rpc error: code = NotFound desc = could not find container \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": container with ID starting with 987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471 not found: ID does not exist"
Jul 21 16:30:56.770710 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.770677    2580 scope.go:117] "RemoveContainer" containerID="35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"
Jul 21 16:30:56.770910 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:30:56.770894    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": container with ID starting with 35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd not found: ID does not exist" containerID="35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"
Jul 21 16:30:56.770952 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.770915    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"} err="failed to get container status \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": rpc error: code = NotFound desc = could not find container \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": container with ID starting with 35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd not found: ID does not exist"
Jul 21 16:30:56.770952 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.770928    2580 scope.go:117] "RemoveContainer" containerID="fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"
Jul 21 16:30:56.771159 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:30:56.771145    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": container with ID starting with fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf not found: ID does not exist" containerID="fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"
Jul 21 16:30:56.771197 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.771163    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"} err="failed to get container status \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": rpc error: code = NotFound desc = could not find container \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": container with ID starting with fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf not found: ID does not exist"
Jul 21 16:30:56.771197 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.771176    2580 scope.go:117] "RemoveContainer" containerID="dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"
Jul 21 16:30:56.771388 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:30:56.771372    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": container with ID starting with dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634 not found: ID does not exist" containerID="dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"
Jul 21 16:30:56.771422 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.771393    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"} err="failed to get container status \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": rpc error: code = NotFound desc = could not find container \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": container with ID starting with dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634 not found: ID does not exist"
Jul 21 16:30:56.771422 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.771406    2580 scope.go:117] "RemoveContainer" containerID="3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"
Jul 21 16:30:56.771633 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.771613    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"} err="failed to get container status \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": rpc error: code = NotFound desc = could not find container \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": container with ID starting with 3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab not found: ID does not exist"
Jul 21 16:30:56.771677 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.771635    2580 scope.go:117] "RemoveContainer" containerID="6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"
Jul 21 16:30:56.771860 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.771840    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"} err="failed to get container status \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": rpc error: code = NotFound desc = could not find container \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": container with ID starting with 6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8 not found: ID does not exist"
Jul 21 16:30:56.771860 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.771859    2580 scope.go:117] "RemoveContainer" containerID="fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"
Jul 21 16:30:56.772084 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.772066    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"} err="failed to get container status \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": rpc error: code = NotFound desc = could not find container \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": container with ID starting with fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57 not found: ID does not exist"
Jul 21 16:30:56.772158 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.772085    2580 scope.go:117] "RemoveContainer" containerID="987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"
Jul 21 16:30:56.772292 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.772276    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"} err="failed to get container status \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": rpc error: code = NotFound desc = could not find container \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": container with ID starting with 987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471 not found: ID does not exist"
Jul 21 16:30:56.772328 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.772293    2580 scope.go:117] "RemoveContainer" containerID="35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"
Jul 21 16:30:56.772536 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.772496    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"} err="failed to get container status \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": rpc error: code = NotFound desc = could not find container \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": container with ID starting with 35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd not found: ID does not exist"
Jul 21 16:30:56.772536 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.772531    2580 scope.go:117] "RemoveContainer" containerID="fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"
Jul 21 16:30:56.772747 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.772725    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"} err="failed to get container status \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": rpc error: code = NotFound desc = could not find container \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": container with ID starting with fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf not found: ID does not exist"
Jul 21 16:30:56.772795 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.772749    2580 scope.go:117] "RemoveContainer" containerID="dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"
Jul 21 16:30:56.772939 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.772923    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"} err="failed to get container status \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": rpc error: code = NotFound desc = could not find container \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": container with ID starting with dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634 not found: ID does not exist"
Jul 21 16:30:56.772985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.772941    2580 scope.go:117] "RemoveContainer" containerID="3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"
Jul 21 16:30:56.773114 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.773099    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"} err="failed to get container status \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": rpc error: code = NotFound desc = could not find container \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": container with ID starting with 3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab not found: ID does not exist"
Jul 21 16:30:56.773156 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.773113    2580 scope.go:117] "RemoveContainer" containerID="6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"
Jul 21 16:30:56.773294 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.773277    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"} err="failed to get container status \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": rpc error: code = NotFound desc = could not find container \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": container with ID starting with 6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8 not found: ID does not exist"
Jul 21 16:30:56.773347 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.773295    2580 scope.go:117] "RemoveContainer" containerID="fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"
Jul 21 16:30:56.773484 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.773464    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"} err="failed to get container status \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": rpc error: code = NotFound desc = could not find container \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": container with ID starting with fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57 not found: ID does not exist"
Jul 21 16:30:56.773572 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.773486    2580 scope.go:117] "RemoveContainer" containerID="987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"
Jul 21 16:30:56.773738 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.773720    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"} err="failed to get container status \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": rpc error: code = NotFound desc = could not find container \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": container with ID starting with 987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471 not found: ID does not exist"
Jul 21 16:30:56.773738 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.773737    2580 scope.go:117] "RemoveContainer" containerID="35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"
Jul 21 16:30:56.773946 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.773931    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"} err="failed to get container status \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": rpc error: code = NotFound desc = could not find container \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": container with ID starting with 35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd not found: ID does not exist"
Jul 21 16:30:56.773991 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.773947    2580 scope.go:117] "RemoveContainer" containerID="fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"
Jul 21 16:30:56.774141 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.774126    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"} err="failed to get container status \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": rpc error: code = NotFound desc = could not find container \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": container with ID starting with fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf not found: ID does not exist"
Jul 21 16:30:56.774190 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.774142    2580 scope.go:117] "RemoveContainer" containerID="dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"
Jul 21 16:30:56.774335 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.774317    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"} err="failed to get container status \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": rpc error: code = NotFound desc = could not find container \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": container with ID starting with dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634 not found: ID does not exist"
Jul 21 16:30:56.774402 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.774337    2580 scope.go:117] "RemoveContainer" containerID="3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"
Jul 21 16:30:56.774590 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.774569    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"} err="failed to get container status \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": rpc error: code = NotFound desc = could not find container \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": container with ID starting with 3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab not found: ID does not exist"
Jul 21 16:30:56.774646 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.774590    2580 scope.go:117] "RemoveContainer" containerID="6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"
Jul 21 16:30:56.774780 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.774765    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"} err="failed to get container status \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": rpc error: code = NotFound desc = could not find container \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": container with ID starting with 6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8 not found: ID does not exist"
Jul 21 16:30:56.774824 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.774779    2580 scope.go:117] "RemoveContainer" containerID="fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"
Jul 21 16:30:56.774981 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.774963    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"} err="failed to get container status \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": rpc error: code = NotFound desc = could not find container \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": container with ID starting with fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57 not found: ID does not exist"
Jul 21 16:30:56.775053 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.774983    2580 scope.go:117] "RemoveContainer" containerID="987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"
Jul 21 16:30:56.775184 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.775168    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"} err="failed to get container status \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": rpc error: code = NotFound desc = could not find container \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": container with ID starting with 987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471 not found: ID does not exist"
Jul 21 16:30:56.775229 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.775184    2580 scope.go:117] "RemoveContainer" containerID="35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"
Jul 21 16:30:56.775427 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.775409    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"} err="failed to get container status \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": rpc error: code = NotFound desc = could not find container \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": container with ID starting with 35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd not found: ID does not exist"
Jul 21 16:30:56.775504 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.775429    2580 scope.go:117] "RemoveContainer" containerID="fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"
Jul 21 16:30:56.775858 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.775830    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"} err="failed to get container status \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": rpc error: code = NotFound desc = could not find container \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": container with ID starting with fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf not found: ID does not exist"
Jul 21 16:30:56.775963 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.775858    2580 scope.go:117] "RemoveContainer" containerID="dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"
Jul 21 16:30:56.776320 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.776297    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"} err="failed to get container status \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": rpc error: code = NotFound desc = could not find container \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": container with ID starting with dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634 not found: ID does not exist"
Jul 21 16:30:56.776320 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.776320    2580 scope.go:117] "RemoveContainer" containerID="3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"
Jul 21 16:30:56.776570 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.776549    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"} err="failed to get container status \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": rpc error: code = NotFound desc = could not find container \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": container with ID starting with 3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab not found: ID does not exist"
Jul 21 16:30:56.776640 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.776573    2580 scope.go:117] "RemoveContainer" containerID="6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"
Jul 21 16:30:56.776810 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.776795    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"} err="failed to get container status \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": rpc error: code = NotFound desc = could not find container \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": container with ID starting with 6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8 not found: ID does not exist"
Jul 21 16:30:56.776864 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.776810    2580 scope.go:117] "RemoveContainer" containerID="fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"
Jul 21 16:30:56.777025 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777010    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"} err="failed to get container status \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": rpc error: code = NotFound desc = could not find container \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": container with ID starting with fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57 not found: ID does not exist"
Jul 21 16:30:56.777077 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777024    2580 scope.go:117] "RemoveContainer" containerID="987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"
Jul 21 16:30:56.777262 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777244    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"} err="failed to get container status \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": rpc error: code = NotFound desc = could not find container \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": container with ID starting with 987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471 not found: ID does not exist"
Jul 21 16:30:56.777316 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777263    2580 scope.go:117] "RemoveContainer" containerID="35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"
Jul 21 16:30:56.777422 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777393    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 16:30:56.777495 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777474    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"} err="failed to get container status \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": rpc error: code = NotFound desc = could not find container \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": container with ID starting with 35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd not found: ID does not exist"
Jul 21 16:30:56.777495 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777492    2580 scope.go:117] "RemoveContainer" containerID="fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"
Jul 21 16:30:56.777687 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777674    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="prometheus"
Jul 21 16:30:56.777737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777689    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="prometheus"
Jul 21 16:30:56.777737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777697    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="thanos-sidecar"
Jul 21 16:30:56.777737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777702    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="thanos-sidecar"
Jul 21 16:30:56.777737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777712    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy-web"
Jul 21 16:30:56.777737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777718    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy-web"
Jul 21 16:30:56.777737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777725    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="init-config-reloader"
Jul 21 16:30:56.777737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777729    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="init-config-reloader"
Jul 21 16:30:56.777737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777734    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="config-reloader"
Jul 21 16:30:56.777737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777731    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"} err="failed to get container status \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": rpc error: code = NotFound desc = could not find container \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": container with ID starting with fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf not found: ID does not exist"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777747    2580 scope.go:117] "RemoveContainer" containerID="dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777740    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="config-reloader"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777794    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777802    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777812    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy-thanos"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777820    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy-thanos"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777900    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="prometheus"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777913    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="config-reloader"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777922    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy-web"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777930    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy-thanos"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777940    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="kube-rbac-proxy"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777948    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a030a99-a173-4060-835a-6b344c11ca82" containerName="thanos-sidecar"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.777998    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"} err="failed to get container status \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": rpc error: code = NotFound desc = could not find container \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": container with ID starting with dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634 not found: ID does not exist"
Jul 21 16:30:56.778056 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.778014    2580 scope.go:117] "RemoveContainer" containerID="3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"
Jul 21 16:30:56.778590 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.778213    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab"} err="failed to get container status \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": rpc error: code = NotFound desc = could not find container \"3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab\": container with ID starting with 3db7d23454c1682f4d435e521ec93b23fe8b10f84a28d8f2fd611914de20b0ab not found: ID does not exist"
Jul 21 16:30:56.778590 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.778233    2580 scope.go:117] "RemoveContainer" containerID="6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"
Jul 21 16:30:56.778590 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.778468    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8"} err="failed to get container status \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": rpc error: code = NotFound desc = could not find container \"6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8\": container with ID starting with 6a5626cdc876997c96e42f3afce8f4f4d504540d7e5c0815e18dc542c663f4e8 not found: ID does not exist"
Jul 21 16:30:56.778590 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.778483    2580 scope.go:117] "RemoveContainer" containerID="fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"
Jul 21 16:30:56.778772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.778721    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57"} err="failed to get container status \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": rpc error: code = NotFound desc = could not find container \"fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57\": container with ID starting with fd58129bef433b2dd33d02dff5cb145760da6300326070fffa2e9d8b5a6a7d57 not found: ID does not exist"
Jul 21 16:30:56.778772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.778733    2580 scope.go:117] "RemoveContainer" containerID="987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"
Jul 21 16:30:56.778936 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.778921    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471"} err="failed to get container status \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": rpc error: code = NotFound desc = could not find container \"987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471\": container with ID starting with 987749c8217ab669cb7a1d92de659fcb022178a121c6dea760d19614ad3e4471 not found: ID does not exist"
Jul 21 16:30:56.778936 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.778935    2580 scope.go:117] "RemoveContainer" containerID="35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"
Jul 21 16:30:56.779180 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.779163    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd"} err="failed to get container status \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": rpc error: code = NotFound desc = could not find container \"35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd\": container with ID starting with 35f525d292873c8e6347004f49cb8e26774ba39da737142c20c2f46d0c0b98cd not found: ID does not exist"
Jul 21 16:30:56.779180 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.779180    2580 scope.go:117] "RemoveContainer" containerID="fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"
Jul 21 16:30:56.779412 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.779394    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf"} err="failed to get container status \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": rpc error: code = NotFound desc = could not find container \"fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf\": container with ID starting with fd6d00e17df9d818d40b6edac0cb381f9848c1f281c32956284be7fa331929bf not found: ID does not exist"
Jul 21 16:30:56.779412 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.779411    2580 scope.go:117] "RemoveContainer" containerID="dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"
Jul 21 16:30:56.779654 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.779630    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634"} err="failed to get container status \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": rpc error: code = NotFound desc = could not find container \"dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634\": container with ID starting with dd792b441ce5a8114b69822f81107f84e5ae2cfd4e04b7a1ac131dae75ac0634 not found: ID does not exist"
Jul 21 16:30:56.783116 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.783097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.785546 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.785506    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 21 16:30:56.785718 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.785698    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 21 16:30:56.785802 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.785760    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 21 16:30:56.785802 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.785788    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9ea30dcq5tg7r\""
Jul 21 16:30:56.785912 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.785883    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 21 16:30:56.785912 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.785891    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 21 16:30:56.786206 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.786189    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 21 16:30:56.786453 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.786437    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 21 16:30:56.786891 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.786823    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 21 16:30:56.786997 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.786899    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 21 16:30:56.787055 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.786994    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 21 16:30:56.787055 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.787040    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 21 16:30:56.787210 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.787189    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 21 16:30:56.787427 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.787411    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qxr25\""
Jul 21 16:30:56.789273 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.789252    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 21 16:30:56.791695 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.791679    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 21 16:30:56.796619 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.796602    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 16:30:56.870375 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x87t\" (UniqueName: \"kubernetes.io/projected/ea6de6a3-750e-44e9-970f-3ce857090c0d-kube-api-access-7x87t\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea6de6a3-750e-44e9-970f-3ce857090c0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-config\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870603    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.870851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.871239 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.871239 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.870911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea6de6a3-750e-44e9-970f-3ce857090c0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.938701 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.938670    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a030a99-a173-4060-835a-6b344c11ca82" path="/var/lib/kubelet/pods/4a030a99-a173-4060-835a-6b344c11ca82/volumes"
Jul 21 16:30:56.971196 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x87t\" (UniqueName: \"kubernetes.io/projected/ea6de6a3-750e-44e9-970f-3ce857090c0d-kube-api-access-7x87t\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971196 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971355 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea6de6a3-750e-44e9-970f-3ce857090c0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971355 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971355 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971268    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971355 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971355 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971626 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-config\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971626 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971626 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971626 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.971626 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.971471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972114 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972207 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972207 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972207 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972391 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972391 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972391 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972391 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972391 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972391 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972345    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea6de6a3-750e-44e9-970f-3ce857090c0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972866 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.972966 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.972941    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.973388 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.973106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.974155 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.974020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.974590 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.974563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.974689 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.974658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea6de6a3-750e-44e9-970f-3ce857090c0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.975275 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.975253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.975418 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.975341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.975903 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.975768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.975903 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.975866    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.976206 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.976174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.976740 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.976719    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.977120 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.977101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-config\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.977371 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.977333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea6de6a3-750e-44e9-970f-3ce857090c0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.977685 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.977657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.977992 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.977929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6de6a3-750e-44e9-970f-3ce857090c0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.978064 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.978026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea6de6a3-750e-44e9-970f-3ce857090c0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:56.979435 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:56.979416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x87t\" (UniqueName: \"kubernetes.io/projected/ea6de6a3-750e-44e9-970f-3ce857090c0d-kube-api-access-7x87t\") pod \"prometheus-k8s-0\" (UID: \"ea6de6a3-750e-44e9-970f-3ce857090c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:57.093686 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:57.093656    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:30:57.234914 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:57.234853    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 21 16:30:57.237983 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:30:57.237959    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6de6a3_750e_44e9_970f_3ce857090c0d.slice/crio-d0052bdaedbe630bee35eba0a22861080e9d1944762bb11042d2623c71897fb5 WatchSource:0}: Error finding container d0052bdaedbe630bee35eba0a22861080e9d1944762bb11042d2623c71897fb5: Status 404 returned error can't find the container with id d0052bdaedbe630bee35eba0a22861080e9d1944762bb11042d2623c71897fb5
Jul 21 16:30:57.731793 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:57.731751    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea6de6a3-750e-44e9-970f-3ce857090c0d" containerID="326ac7dbc8efe06a9d44a972ca3daa2878d3c0fe6dcef5d946adf78d668b453d" exitCode=0
Jul 21 16:30:57.731958 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:57.731824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6de6a3-750e-44e9-970f-3ce857090c0d","Type":"ContainerDied","Data":"326ac7dbc8efe06a9d44a972ca3daa2878d3c0fe6dcef5d946adf78d668b453d"}
Jul 21 16:30:57.731958 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:57.731908    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6de6a3-750e-44e9-970f-3ce857090c0d","Type":"ContainerStarted","Data":"d0052bdaedbe630bee35eba0a22861080e9d1944762bb11042d2623c71897fb5"}
Jul 21 16:30:58.740318 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:58.740286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6de6a3-750e-44e9-970f-3ce857090c0d","Type":"ContainerStarted","Data":"816a969ac1d07e742a62907ae9d279fcd44a8cee24ec1e9140ea102d8e0a4a97"}
Jul 21 16:30:58.740792 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:58.740347    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6de6a3-750e-44e9-970f-3ce857090c0d","Type":"ContainerStarted","Data":"b6621cd71e226c40c20d7306fb2933f3c9b58a2ee4c37041e309390f0b883fc6"}
Jul 21 16:30:58.740792 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:58.740374    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6de6a3-750e-44e9-970f-3ce857090c0d","Type":"ContainerStarted","Data":"4362b660bd724c66d8ba8d6cf978ecd5ca74d5218410b727dc01e7f8704d7d19"}
Jul 21 16:30:58.740792 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:58.740399    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6de6a3-750e-44e9-970f-3ce857090c0d","Type":"ContainerStarted","Data":"b984c28ffc31596190b830cc40f5b20a03ba33a1c39c401f5f37b374087a3bb6"}
Jul 21 16:30:58.740792 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:58.740424    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6de6a3-750e-44e9-970f-3ce857090c0d","Type":"ContainerStarted","Data":"8523ddaaae31a1e0098ba3626ca90b4c1b24423f9aef35d9b791d819e931dee5"}
Jul 21 16:30:58.740792 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:58.740446    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6de6a3-750e-44e9-970f-3ce857090c0d","Type":"ContainerStarted","Data":"084aaac27fc5867e007c0fd93fbe5794224b83810d52a932298af3f0e0280bb7"}
Jul 21 16:30:58.773231 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:30:58.773183    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.7530798130000003 podStartE2EDuration="2.773167098s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.087285ms" isStatefulPod=false podCreationTimestamp="2026-07-21 16:30:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:30:58.770257904 +0000 UTC m=+154.269619189" watchObservedRunningTime="2026-07-21 16:30:58.773167098 +0000 UTC m=+154.272528342"
Jul 21 16:31:02.094443 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:02.094406    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:31:03.343281 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:03.343246    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:31:03.343745 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:03.343306    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:31:03.348176 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:03.348157    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:31:03.763036 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:03.763003    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:31:03.805556 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:03.805526    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dfbc9b98b-rqfc8"]
Jul 21 16:31:28.831379 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:28.831340    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-dfbc9b98b-rqfc8" podUID="1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" containerName="console" containerID="cri-o://4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442" gracePeriod=15
Jul 21 16:31:29.078634 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.078614    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:31:29.138907 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.138827    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-serving-cert\") pod \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") "
Jul 21 16:31:29.138907 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.138872    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-oauth-serving-cert\") pod \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") "
Jul 21 16:31:29.138907 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.138891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-service-ca\") pod \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") "
Jul 21 16:31:29.139171 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.138931    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-trusted-ca-bundle\") pod \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") "
Jul 21 16:31:29.139171 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.138962    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-config\") pod \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") "
Jul 21 16:31:29.139171 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.139101    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kp7h\" (UniqueName: \"kubernetes.io/projected/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-kube-api-access-5kp7h\") pod \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") "
Jul 21 16:31:29.139171 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.139144    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-oauth-config\") pod \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\" (UID: \"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a\") "
Jul 21 16:31:29.139367 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.139317    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" (UID: "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:31:29.139424 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.139379    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-oauth-serving-cert\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:31:29.139476 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.139437    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-config" (OuterVolumeSpecName: "console-config") pod "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" (UID: "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:31:29.139476 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.139447    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-service-ca" (OuterVolumeSpecName: "service-ca") pod "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" (UID: "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:31:29.139599 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.139485    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" (UID: "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:31:29.141158 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.141133    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" (UID: "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:31:29.141319 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.141295    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-kube-api-access-5kp7h" (OuterVolumeSpecName: "kube-api-access-5kp7h") pod "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" (UID: "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a"). InnerVolumeSpecName "kube-api-access-5kp7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:31:29.141372 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.141326    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" (UID: "1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:31:29.239764 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.239737    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:31:29.239764 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.239764    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kp7h\" (UniqueName: \"kubernetes.io/projected/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-kube-api-access-5kp7h\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:31:29.239923 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.239778    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-oauth-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:31:29.239923 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.239790    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-console-serving-cert\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:31:29.239923 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.239803    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:31:29.239923 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.239815    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a-trusted-ca-bundle\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:31:29.859693 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.859652    2580 generic.go:356] "Generic (PLEG): container finished" podID="1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" containerID="4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442" exitCode=0
Jul 21 16:31:29.859693 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.859694    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfbc9b98b-rqfc8"
Jul 21 16:31:29.860232 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.859732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfbc9b98b-rqfc8" event={"ID":"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a","Type":"ContainerDied","Data":"4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442"}
Jul 21 16:31:29.860232 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.859776    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfbc9b98b-rqfc8" event={"ID":"1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a","Type":"ContainerDied","Data":"8e89dcba0e3cbf0106e4774e03f7af949e05f22a109bca95f3c3b787514d189b"}
Jul 21 16:31:29.860232 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.859800    2580 scope.go:117] "RemoveContainer" containerID="4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442"
Jul 21 16:31:29.868795 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.868781    2580 scope.go:117] "RemoveContainer" containerID="4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442"
Jul 21 16:31:29.869075 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:31:29.869055    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442\": container with ID starting with 4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442 not found: ID does not exist" containerID="4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442"
Jul 21 16:31:29.869124 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.869082    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442"} err="failed to get container status \"4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442\": rpc error: code = NotFound desc = could not find container \"4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442\": container with ID starting with 4dfe481c807787de65f8bfa38054fe3b418c76349a232aa2c19dee52ff30a442 not found: ID does not exist"
Jul 21 16:31:29.880705 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.880683    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dfbc9b98b-rqfc8"]
Jul 21 16:31:29.884805 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:29.884786    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dfbc9b98b-rqfc8"]
Jul 21 16:31:30.938048 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:30.938016    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" path="/var/lib/kubelet/pods/1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a/volumes"
Jul 21 16:31:57.093826 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:57.093792    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:31:57.110351 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:57.110327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:31:57.988482 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:31:57.988454    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 21 16:32:24.443210 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.443175    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wsxhj"]
Jul 21 16:32:24.443668 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.443441    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" containerName="console"
Jul 21 16:32:24.443668 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.443453    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" containerName="console"
Jul 21 16:32:24.443668 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.443507    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef1e3d4-0161-4006-9a25-5cf98ce2ae3a" containerName="console"
Jul 21 16:32:24.446459 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.446442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wsxhj"
Jul 21 16:32:24.453027 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.453001    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-czmg5\""
Jul 21 16:32:24.453299 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.453284    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 21 16:32:24.453966 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.453949    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 21 16:32:24.463845 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.463823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wsxhj"]
Jul 21 16:32:24.503004 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.502978    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtqx7\" (UniqueName: \"kubernetes.io/projected/3b36ce61-fc56-416c-b911-22ce621cb7fd-kube-api-access-mtqx7\") pod \"cert-manager-55b4d954d5-wsxhj\" (UID: \"3b36ce61-fc56-416c-b911-22ce621cb7fd\") " pod="cert-manager/cert-manager-55b4d954d5-wsxhj"
Jul 21 16:32:24.503136 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.503022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b36ce61-fc56-416c-b911-22ce621cb7fd-bound-sa-token\") pod \"cert-manager-55b4d954d5-wsxhj\" (UID: \"3b36ce61-fc56-416c-b911-22ce621cb7fd\") " pod="cert-manager/cert-manager-55b4d954d5-wsxhj"
Jul 21 16:32:24.603747 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.603710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtqx7\" (UniqueName: \"kubernetes.io/projected/3b36ce61-fc56-416c-b911-22ce621cb7fd-kube-api-access-mtqx7\") pod \"cert-manager-55b4d954d5-wsxhj\" (UID: \"3b36ce61-fc56-416c-b911-22ce621cb7fd\") " pod="cert-manager/cert-manager-55b4d954d5-wsxhj"
Jul 21 16:32:24.603900 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.603773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b36ce61-fc56-416c-b911-22ce621cb7fd-bound-sa-token\") pod \"cert-manager-55b4d954d5-wsxhj\" (UID: \"3b36ce61-fc56-416c-b911-22ce621cb7fd\") " pod="cert-manager/cert-manager-55b4d954d5-wsxhj"
Jul 21 16:32:24.620930 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.620898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtqx7\" (UniqueName: \"kubernetes.io/projected/3b36ce61-fc56-416c-b911-22ce621cb7fd-kube-api-access-mtqx7\") pod \"cert-manager-55b4d954d5-wsxhj\" (UID: \"3b36ce61-fc56-416c-b911-22ce621cb7fd\") " pod="cert-manager/cert-manager-55b4d954d5-wsxhj"
Jul 21 16:32:24.621086 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.621070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b36ce61-fc56-416c-b911-22ce621cb7fd-bound-sa-token\") pod \"cert-manager-55b4d954d5-wsxhj\" (UID: \"3b36ce61-fc56-416c-b911-22ce621cb7fd\") " pod="cert-manager/cert-manager-55b4d954d5-wsxhj"
Jul 21 16:32:24.771214 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.771136    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wsxhj"
Jul 21 16:32:24.926047 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:24.926027    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wsxhj"]
Jul 21 16:32:24.928261 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:32:24.928233    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b36ce61_fc56_416c_b911_22ce621cb7fd.slice/crio-d8924b3e9f5f3adf0a257de29ac508444e49db5a7d2c68333f8e1b7b332a1fe7 WatchSource:0}: Error finding container d8924b3e9f5f3adf0a257de29ac508444e49db5a7d2c68333f8e1b7b332a1fe7: Status 404 returned error can't find the container with id d8924b3e9f5f3adf0a257de29ac508444e49db5a7d2c68333f8e1b7b332a1fe7
Jul 21 16:32:25.073283 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:25.073217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wsxhj" event={"ID":"3b36ce61-fc56-416c-b911-22ce621cb7fd","Type":"ContainerStarted","Data":"d8924b3e9f5f3adf0a257de29ac508444e49db5a7d2c68333f8e1b7b332a1fe7"}
Jul 21 16:32:28.086842 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:28.086807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wsxhj" event={"ID":"3b36ce61-fc56-416c-b911-22ce621cb7fd","Type":"ContainerStarted","Data":"a97eb8eeb14c7bb940265e43b203f4e7892735e90d1e86201b3051f13e2d486a"}
Jul 21 16:32:48.384898 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.384807    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wsxhj" podStartSLOduration=21.382389221 podStartE2EDuration="24.384791048s" totalImagesPullingTime="3.002401827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:32:28.121434357 +0000 UTC m=+243.620795601" watchObservedRunningTime="2026-07-21 16:32:48.384791048 +0000 UTC m=+263.884152356"
Jul 21 16:32:48.385256 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.384993    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"]
Jul 21 16:32:48.388864 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.388844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.394174 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:32:48.394142    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" is forbidden: User \"system:node:ip-10-0-141-158.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-158.ec2.internal' and this object" podUID="aec37a82-46f6-46a3-8c24-37985db8b9d1" pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.395212 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.395175    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 16:32:48.395499 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.395358    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 16:32:48.395616 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.395493    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 16:32:48.395806 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.395751    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2bwxz\""
Jul 21 16:32:48.396105 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.396068    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 16:32:48.422774 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.422742    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"]
Jul 21 16:32:48.479657 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.479636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aec37a82-46f6-46a3-8c24-37985db8b9d1-manifests\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.479772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.479679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnmmf\" (UniqueName: \"kubernetes.io/projected/aec37a82-46f6-46a3-8c24-37985db8b9d1-kube-api-access-bnmmf\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.479772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.479725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aec37a82-46f6-46a3-8c24-37985db8b9d1-tmp\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.479772 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.479746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aec37a82-46f6-46a3-8c24-37985db8b9d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.479931 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.479776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aec37a82-46f6-46a3-8c24-37985db8b9d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.580974 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.580950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aec37a82-46f6-46a3-8c24-37985db8b9d1-manifests\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.581063 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.580992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnmmf\" (UniqueName: \"kubernetes.io/projected/aec37a82-46f6-46a3-8c24-37985db8b9d1-kube-api-access-bnmmf\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.581063 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.581034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aec37a82-46f6-46a3-8c24-37985db8b9d1-tmp\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.581063 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.581051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aec37a82-46f6-46a3-8c24-37985db8b9d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.581202 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.581076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aec37a82-46f6-46a3-8c24-37985db8b9d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.581308 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.581289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aec37a82-46f6-46a3-8c24-37985db8b9d1-manifests\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.583548 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.583502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aec37a82-46f6-46a3-8c24-37985db8b9d1-tmp\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.583869 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.583848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aec37a82-46f6-46a3-8c24-37985db8b9d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.583918 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.583848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aec37a82-46f6-46a3-8c24-37985db8b9d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.601892 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.601867    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnmmf\" (UniqueName: \"kubernetes.io/projected/aec37a82-46f6-46a3-8c24-37985db8b9d1-kube-api-access-bnmmf\") pod \"opendatahub-operator-controller-manager-87c9445c4-n6cdx\" (UID: \"aec37a82-46f6-46a3-8c24-37985db8b9d1\") " pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.703962 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.703943    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:48.852677 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:48.852651    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"]
Jul 21 16:32:48.855441 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:32:48.855414    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec37a82_46f6_46a3_8c24_37985db8b9d1.slice/crio-77661d14206e0b59a1401e0cd45d6d9f1644747ada72f7d440669d7f1b506766 WatchSource:0}: Error finding container 77661d14206e0b59a1401e0cd45d6d9f1644747ada72f7d440669d7f1b506766: Status 404 returned error can't find the container with id 77661d14206e0b59a1401e0cd45d6d9f1644747ada72f7d440669d7f1b506766
Jul 21 16:32:49.167354 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:49.167320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx" event={"ID":"aec37a82-46f6-46a3-8c24-37985db8b9d1","Type":"ContainerStarted","Data":"77661d14206e0b59a1401e0cd45d6d9f1644747ada72f7d440669d7f1b506766"}
Jul 21 16:32:52.180634 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:52.180602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx" event={"ID":"aec37a82-46f6-46a3-8c24-37985db8b9d1","Type":"ContainerStarted","Data":"2c664add0e763b2df44b07dc08ad05468abb3822f0d1752bf2261c5e2c56a35e"}
Jul 21 16:32:53.185834 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:53.185802    2580 generic.go:356] "Generic (PLEG): container finished" podID="aec37a82-46f6-46a3-8c24-37985db8b9d1" containerID="2c664add0e763b2df44b07dc08ad05468abb3822f0d1752bf2261c5e2c56a35e" exitCode=0
Jul 21 16:32:53.186201 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:53.185893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx" event={"ID":"aec37a82-46f6-46a3-8c24-37985db8b9d1","Type":"ContainerDied","Data":"2c664add0e763b2df44b07dc08ad05468abb3822f0d1752bf2261c5e2c56a35e"}
Jul 21 16:32:56.200473 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:56.200402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx" event={"ID":"aec37a82-46f6-46a3-8c24-37985db8b9d1","Type":"ContainerStarted","Data":"446783c94ed18c67f0bafe81c70c7bf540a414e7f450398074bf4fbceec47b4e"}
Jul 21 16:32:56.200473 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:56.200471    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:32:56.227641 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:32:56.227593    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx" podStartSLOduration=1.6097981 podStartE2EDuration="8.227560793s" totalImagesPullingTime="5.626508524s" totalInitContainerRuntime="991.254169ms" isStatefulPod=false podCreationTimestamp="2026-07-21 16:32:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:32:56.226187767 +0000 UTC m=+271.725549002" watchObservedRunningTime="2026-07-21 16:32:56.227560793 +0000 UTC m=+271.726922112"
Jul 21 16:33:07.207135 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:07.207109    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-87c9445c4-n6cdx"
Jul 21 16:33:11.703583 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.703549    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"]
Jul 21 16:33:11.712296 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.712271    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"]
Jul 21 16:33:11.712419 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.712364    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:11.716451 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.716429    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7rslk\""
Jul 21 16:33:11.716758 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.716741    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 16:33:11.717235 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.717220    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 16:33:11.717328 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.717302    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 21 16:33:11.717395 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.717345    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 16:33:11.755296 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.755268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc7zk\" (UniqueName: \"kubernetes.io/projected/cd7875a7-f877-4a29-aa5f-6f3dc41ebb35-kube-api-access-gc7zk\") pod \"kube-auth-proxy-7d77f95f8-tx4tj\" (UID: \"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35\") " pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:11.755296 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.755297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd7875a7-f877-4a29-aa5f-6f3dc41ebb35-tmp\") pod \"kube-auth-proxy-7d77f95f8-tx4tj\" (UID: \"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35\") " pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:11.755436 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.755313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd7875a7-f877-4a29-aa5f-6f3dc41ebb35-tls-certs\") pod \"kube-auth-proxy-7d77f95f8-tx4tj\" (UID: \"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35\") " pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:11.855838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.855806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc7zk\" (UniqueName: \"kubernetes.io/projected/cd7875a7-f877-4a29-aa5f-6f3dc41ebb35-kube-api-access-gc7zk\") pod \"kube-auth-proxy-7d77f95f8-tx4tj\" (UID: \"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35\") " pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:11.855838 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.855840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd7875a7-f877-4a29-aa5f-6f3dc41ebb35-tmp\") pod \"kube-auth-proxy-7d77f95f8-tx4tj\" (UID: \"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35\") " pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:11.856022 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.855860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd7875a7-f877-4a29-aa5f-6f3dc41ebb35-tls-certs\") pod \"kube-auth-proxy-7d77f95f8-tx4tj\" (UID: \"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35\") " pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:11.858141 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.858114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd7875a7-f877-4a29-aa5f-6f3dc41ebb35-tmp\") pod \"kube-auth-proxy-7d77f95f8-tx4tj\" (UID: \"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35\") " pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:11.858245 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.858232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd7875a7-f877-4a29-aa5f-6f3dc41ebb35-tls-certs\") pod \"kube-auth-proxy-7d77f95f8-tx4tj\" (UID: \"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35\") " pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:11.883976 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:11.883948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc7zk\" (UniqueName: \"kubernetes.io/projected/cd7875a7-f877-4a29-aa5f-6f3dc41ebb35-kube-api-access-gc7zk\") pod \"kube-auth-proxy-7d77f95f8-tx4tj\" (UID: \"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35\") " pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:12.023441 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:12.023384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"
Jul 21 16:33:12.161797 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:12.161740    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj"]
Jul 21 16:33:12.166050 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:33:12.166024    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd7875a7_f877_4a29_aa5f_6f3dc41ebb35.slice/crio-5e986f19565f3374a487fb52c5c60219c148b5ad0e51a84fbdfa48ea8c37d7f8 WatchSource:0}: Error finding container 5e986f19565f3374a487fb52c5c60219c148b5ad0e51a84fbdfa48ea8c37d7f8: Status 404 returned error can't find the container with id 5e986f19565f3374a487fb52c5c60219c148b5ad0e51a84fbdfa48ea8c37d7f8
Jul 21 16:33:12.260074 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:12.260038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj" event={"ID":"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35","Type":"ContainerStarted","Data":"5e986f19565f3374a487fb52c5c60219c148b5ad0e51a84fbdfa48ea8c37d7f8"}
Jul 21 16:33:15.275206 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:15.275169    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj" event={"ID":"cd7875a7-f877-4a29-aa5f-6f3dc41ebb35","Type":"ContainerStarted","Data":"a4ca91054124cdc20f1b8fc88a0e8350d88342dd7a71819269a7f549d8d64f17"}
Jul 21 16:33:19.685906 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.685829    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d77f95f8-tx4tj" podStartSLOduration=6.330874822 podStartE2EDuration="8.685812424s" totalImagesPullingTime="2.354937602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:33:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:33:15.405323405 +0000 UTC m=+290.904684650" watchObservedRunningTime="2026-07-21 16:33:19.685812424 +0000 UTC m=+295.185173667"
Jul 21 16:33:19.687044 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.687026    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"]
Jul 21 16:33:19.690571 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.690557    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.693139 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.693119    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xsjzj\""
Jul 21 16:33:19.693263 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.693244    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 16:33:19.693340 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.693324    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 16:33:19.693526 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.693498    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 16:33:19.693740 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.693725    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 16:33:19.703870 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.703850    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"]
Jul 21 16:33:19.716747 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.716726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.716872 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.716777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.716872 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.716838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t2lj\" (UniqueName: \"kubernetes.io/projected/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-kube-api-access-9t2lj\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.716872 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.716864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.717025 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.716917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.717025 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.716979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.717025 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.717010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.817384 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.817357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.817602 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.817409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.817602 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.817446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.817602 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.817471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.817602 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.817540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.817602 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.817597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.817959 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.817636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t2lj\" (UniqueName: \"kubernetes.io/projected/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-kube-api-access-9t2lj\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.818407 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.818381    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.819973 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.819948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.820099 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.820080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.820175 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.820154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.820175 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.820159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.829382 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.829360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.829851 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.829829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t2lj\" (UniqueName: \"kubernetes.io/projected/2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d-kube-api-access-9t2lj\") pod \"istiod-openshift-gateway-94bb8fbfd-mk8pp\" (UID: \"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:19.840321 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.840298    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"]
Jul 21 16:33:19.843873 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.843860    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:19.846245 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.846228    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 21 16:33:19.846639 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.846622    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 21 16:33:19.846715 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.846628    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 21 16:33:19.846715 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.846702    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 21 16:33:19.846890 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.846871    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5bd79\""
Jul 21 16:33:19.847013 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.846979    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 21 16:33:19.856677 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.856659    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"]
Jul 21 16:33:19.917960 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.917939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-cert\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:19.918071 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.917966    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lwmw\" (UniqueName: \"kubernetes.io/projected/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-kube-api-access-9lwmw\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:19.918071 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.917984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-manager-config\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:19.918151 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.918071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-metrics-cert\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:19.999288 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:19.999229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:20.018782 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.018755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-metrics-cert\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:20.018869 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.018819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-cert\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:20.018869 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.018838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lwmw\" (UniqueName: \"kubernetes.io/projected/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-kube-api-access-9lwmw\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:20.018869 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.018865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-manager-config\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:20.019501 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.019480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-manager-config\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:20.021330 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.021306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-cert\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:20.021564 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.021545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-metrics-cert\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:20.036166 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.036143    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lwmw\" (UniqueName: \"kubernetes.io/projected/9f774a1d-3ddb-45ce-81ca-6fbda981f33d-kube-api-access-9lwmw\") pod \"lws-controller-manager-54b99c46f6-k2r5d\" (UID: \"9f774a1d-3ddb-45ce-81ca-6fbda981f33d\") " pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:20.153269 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.153238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:20.183737 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.183711    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"]
Jul 21 16:33:20.187877 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:33:20.187851    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c1d8c94_0b5d_45a2_a0d9_171ae1fa829d.slice/crio-9d7771a0fa92834696e7d6b85a44d2d439792c31b126f7745a15ecf8ef3c33b5 WatchSource:0}: Error finding container 9d7771a0fa92834696e7d6b85a44d2d439792c31b126f7745a15ecf8ef3c33b5: Status 404 returned error can't find the container with id 9d7771a0fa92834696e7d6b85a44d2d439792c31b126f7745a15ecf8ef3c33b5
Jul 21 16:33:20.298963 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.298898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp" event={"ID":"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d","Type":"ContainerStarted","Data":"9d7771a0fa92834696e7d6b85a44d2d439792c31b126f7745a15ecf8ef3c33b5"}
Jul 21 16:33:20.336103 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:20.336082    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"]
Jul 21 16:33:20.336990 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:33:20.336959    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f774a1d_3ddb_45ce_81ca_6fbda981f33d.slice/crio-d52f277e98b9341510153454615c9a6cc6580115d6cd506f97b2276289cad5f0 WatchSource:0}: Error finding container d52f277e98b9341510153454615c9a6cc6580115d6cd506f97b2276289cad5f0: Status 404 returned error can't find the container with id d52f277e98b9341510153454615c9a6cc6580115d6cd506f97b2276289cad5f0
Jul 21 16:33:21.303553 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:21.303498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d" event={"ID":"9f774a1d-3ddb-45ce-81ca-6fbda981f33d","Type":"ContainerStarted","Data":"d52f277e98b9341510153454615c9a6cc6580115d6cd506f97b2276289cad5f0"}
Jul 21 16:33:24.321068 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:24.321029    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d" event={"ID":"9f774a1d-3ddb-45ce-81ca-6fbda981f33d","Type":"ContainerStarted","Data":"19abf109bc808facf4cc788188ec92d42cb520364cca7281ee17cf03e13ddb65"}
Jul 21 16:33:24.321463 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:24.321182    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:33:24.453196 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:24.453142    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d" podStartSLOduration=1.665434201 podStartE2EDuration="5.45312583s" totalImagesPullingTime="3.787691629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:33:24.452648089 +0000 UTC m=+299.952009339" watchObservedRunningTime="2026-07-21 16:33:24.45312583 +0000 UTC m=+299.952487055"
Jul 21 16:33:24.828968 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:24.828898    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 16:33:27.030930 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:27.030165    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 16:33:27.030930 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:27.030227    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 16:33:27.336493 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:27.336400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp" event={"ID":"2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d","Type":"ContainerStarted","Data":"535e4477bfc2a775147be05cb629d098db5f9e821661c1a06e610ed9b8a11d29"}
Jul 21 16:33:27.336493 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:27.336476    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:27.361475 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:27.361429    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp" podStartSLOduration=1.5218053409999999 podStartE2EDuration="8.36141449s" totalImagesPullingTime="6.839609149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:33:27.359952311 +0000 UTC m=+302.859313554" watchObservedRunningTime="2026-07-21 16:33:27.36141449 +0000 UTC m=+302.860775734"
Jul 21 16:33:28.342457 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:28.342433    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mk8pp"
Jul 21 16:33:35.328029 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:33:35.327993    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-54b99c46f6-k2r5d"
Jul 21 16:34:53.129657 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:34:53.129618    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-889b45655-24ggj"]
Jul 21 16:35:18.150324 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.150279    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-889b45655-24ggj" podUID="83bb5813-e437-4b56-b967-c192cfa1f035" containerName="console" containerID="cri-o://b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d" gracePeriod=15
Jul 21 16:35:18.394785 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.394763    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:35:18.467490 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.467457    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-oauth-config\") pod \"83bb5813-e437-4b56-b967-c192cfa1f035\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") "
Jul 21 16:35:18.467490 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.467491    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54vtp\" (UniqueName: \"kubernetes.io/projected/83bb5813-e437-4b56-b967-c192cfa1f035-kube-api-access-54vtp\") pod \"83bb5813-e437-4b56-b967-c192cfa1f035\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") "
Jul 21 16:35:18.467752 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.467556    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-trusted-ca-bundle\") pod \"83bb5813-e437-4b56-b967-c192cfa1f035\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") "
Jul 21 16:35:18.467752 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.467604    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-oauth-serving-cert\") pod \"83bb5813-e437-4b56-b967-c192cfa1f035\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") "
Jul 21 16:35:18.467752 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.467627    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-serving-cert\") pod \"83bb5813-e437-4b56-b967-c192cfa1f035\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") "
Jul 21 16:35:18.467752 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.467670    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-console-config\") pod \"83bb5813-e437-4b56-b967-c192cfa1f035\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") "
Jul 21 16:35:18.467752 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.467692    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-service-ca\") pod \"83bb5813-e437-4b56-b967-c192cfa1f035\" (UID: \"83bb5813-e437-4b56-b967-c192cfa1f035\") "
Jul 21 16:35:18.468131 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.468091    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "83bb5813-e437-4b56-b967-c192cfa1f035" (UID: "83bb5813-e437-4b56-b967-c192cfa1f035"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:35:18.468243 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.468125    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "83bb5813-e437-4b56-b967-c192cfa1f035" (UID: "83bb5813-e437-4b56-b967-c192cfa1f035"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:35:18.468375 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.468298    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-console-config" (OuterVolumeSpecName: "console-config") pod "83bb5813-e437-4b56-b967-c192cfa1f035" (UID: "83bb5813-e437-4b56-b967-c192cfa1f035"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:35:18.468645 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.468365    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-service-ca" (OuterVolumeSpecName: "service-ca") pod "83bb5813-e437-4b56-b967-c192cfa1f035" (UID: "83bb5813-e437-4b56-b967-c192cfa1f035"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:35:18.469912 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.469883    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "83bb5813-e437-4b56-b967-c192cfa1f035" (UID: "83bb5813-e437-4b56-b967-c192cfa1f035"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:35:18.469912 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.469902    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "83bb5813-e437-4b56-b967-c192cfa1f035" (UID: "83bb5813-e437-4b56-b967-c192cfa1f035"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:35:18.470028 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.469952    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83bb5813-e437-4b56-b967-c192cfa1f035-kube-api-access-54vtp" (OuterVolumeSpecName: "kube-api-access-54vtp") pod "83bb5813-e437-4b56-b967-c192cfa1f035" (UID: "83bb5813-e437-4b56-b967-c192cfa1f035"). InnerVolumeSpecName "kube-api-access-54vtp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:35:18.568279 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.568257    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-oauth-serving-cert\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:35:18.568386 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.568281    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-serving-cert\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:35:18.568386 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.568292    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-console-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:35:18.568386 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.568300    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:35:18.568386 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.568308    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83bb5813-e437-4b56-b967-c192cfa1f035-console-oauth-config\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:35:18.568386 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.568316    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54vtp\" (UniqueName: \"kubernetes.io/projected/83bb5813-e437-4b56-b967-c192cfa1f035-kube-api-access-54vtp\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:35:18.568386 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.568326    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83bb5813-e437-4b56-b967-c192cfa1f035-trusted-ca-bundle\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:35:18.806971 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.806900    2580 generic.go:356] "Generic (PLEG): container finished" podID="83bb5813-e437-4b56-b967-c192cfa1f035" containerID="b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d" exitCode=0
Jul 21 16:35:18.806971 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.806949    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-889b45655-24ggj"
Jul 21 16:35:18.806971 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.806960    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-889b45655-24ggj" event={"ID":"83bb5813-e437-4b56-b967-c192cfa1f035","Type":"ContainerDied","Data":"b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d"}
Jul 21 16:35:18.807157 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.806999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-889b45655-24ggj" event={"ID":"83bb5813-e437-4b56-b967-c192cfa1f035","Type":"ContainerDied","Data":"b208311df4ffacde07b185d17e1c1bb2557fb811c28bc1d1f2774c8a905adc9a"}
Jul 21 16:35:18.807157 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.807018    2580 scope.go:117] "RemoveContainer" containerID="b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d"
Jul 21 16:35:18.815437 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.815421    2580 scope.go:117] "RemoveContainer" containerID="b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d"
Jul 21 16:35:18.815718 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:35:18.815691    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d\": container with ID starting with b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d not found: ID does not exist" containerID="b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d"
Jul 21 16:35:18.815787 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.815716    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d"} err="failed to get container status \"b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d\": rpc error: code = NotFound desc = could not find container \"b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d\": container with ID starting with b3a473dde18742d03f3f983ad691c33646b3b79cf7533ea6e412cbffd285983d not found: ID does not exist"
Jul 21 16:35:18.829034 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.829014    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-889b45655-24ggj"]
Jul 21 16:35:18.834316 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.834296    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-889b45655-24ggj"]
Jul 21 16:35:18.939402 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:35:18.939378    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83bb5813-e437-4b56-b967-c192cfa1f035" path="/var/lib/kubelet/pods/83bb5813-e437-4b56-b967-c192cfa1f035/volumes"
Jul 21 16:37:55.568285 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.568249    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"]
Jul 21 16:37:55.568862 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.568725    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83bb5813-e437-4b56-b967-c192cfa1f035" containerName="console"
Jul 21 16:37:55.568862 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.568743    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="83bb5813-e437-4b56-b967-c192cfa1f035" containerName="console"
Jul 21 16:37:55.568862 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.568828    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="83bb5813-e437-4b56-b967-c192cfa1f035" containerName="console"
Jul 21 16:37:55.571670 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.571650    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.574895 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.574877    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 21 16:37:55.575887 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.575869    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fg4kl\""
Jul 21 16:37:55.575982 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.575930    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 16:37:55.575982 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.575944    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 16:37:55.592144 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.592124    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"]
Jul 21 16:37:55.730424 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.730402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.730568 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.730431    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.730568 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.730452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.730568 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.730471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.730690 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.730570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.730690 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.730614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfx87\" (UniqueName: \"kubernetes.io/projected/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-kube-api-access-pfx87\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.831353 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.831291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.831353 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.831325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.831542 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.831357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.831542 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.831384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.831542 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.831413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.831542 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.831463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfx87\" (UniqueName: \"kubernetes.io/projected/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-kube-api-access-pfx87\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.831740 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.831697    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.831792 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.831780    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.831829 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.831804    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.833582 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.833562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.833942 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.833921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.840767 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.840748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfx87\" (UniqueName: \"kubernetes.io/projected/5b65ccf2-bf4c-4f68-b4eb-9c8556901c08-kube-api-access-pfx87\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb94w\" (UID: \"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:55.881033 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:55.881015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:37:56.012628 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:56.012604    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"]
Jul 21 16:37:56.014599 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:37:56.014576    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b65ccf2_bf4c_4f68_b4eb_9c8556901c08.slice/crio-eff4239b129264fb48c35e4f0ec1c5a5de8e5fda25df06c96225f90e867cd804 WatchSource:0}: Error finding container eff4239b129264fb48c35e4f0ec1c5a5de8e5fda25df06c96225f90e867cd804: Status 404 returned error can't find the container with id eff4239b129264fb48c35e4f0ec1c5a5de8e5fda25df06c96225f90e867cd804
Jul 21 16:37:56.015783 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:56.015759    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 16:37:56.440103 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:56.440068    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w" event={"ID":"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08","Type":"ContainerStarted","Data":"eff4239b129264fb48c35e4f0ec1c5a5de8e5fda25df06c96225f90e867cd804"}
Jul 21 16:37:57.445771 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:57.445737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w" event={"ID":"5b65ccf2-bf4c-4f68-b4eb-9c8556901c08","Type":"ContainerStarted","Data":"eda0be804533e96d74640a27d4e093293ae972dc882a7ef5d7d20ea0da439cf7"}
Jul 21 16:37:57.469726 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:37:57.469679    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w" podStartSLOduration=1.4838157189999999 podStartE2EDuration="2.469664616s" totalImagesPullingTime="985.848897ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:37:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:37:57.46766478 +0000 UTC m=+572.967026023" watchObservedRunningTime="2026-07-21 16:37:57.469664616 +0000 UTC m=+572.969025858"
Jul 21 16:38:05.881462 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:38:05.881432    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:38:05.881910 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:38:05.881535    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:38:05.893556 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:38:05.893536    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:38:06.490677 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:38:06.490646    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb94w"
Jul 21 16:45:00.150419 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:00.150386    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9"]
Jul 21 16:45:00.153690 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:00.153676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9"
Jul 21 16:45:00.156389 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:00.156358    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 16:45:00.163039 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:00.163022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9"
Jul 21 16:45:00.175232 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:00.175207    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9"]
Jul 21 16:45:00.296563 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:00.296537    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9"]
Jul 21 16:45:00.299156 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:45:00.299130    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5d5552_19e6_486e_94bf_c6a91b39b987.slice/crio-2dd71e3686c7336e744eb6b03c460f47743c6ebf23bf53fbee60fd3cf6d228f2 WatchSource:0}: Error finding container 2dd71e3686c7336e744eb6b03c460f47743c6ebf23bf53fbee60fd3cf6d228f2: Status 404 returned error can't find the container with id 2dd71e3686c7336e744eb6b03c460f47743c6ebf23bf53fbee60fd3cf6d228f2
Jul 21 16:45:00.301095 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:00.301079    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 16:45:01.154403 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:01.154367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9" event={"ID":"3a5d5552-19e6-486e-94bf-c6a91b39b987","Type":"ContainerStarted","Data":"2dd71e3686c7336e744eb6b03c460f47743c6ebf23bf53fbee60fd3cf6d228f2"}
Jul 21 16:45:03.166449 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:03.166410    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9" event={"ID":"3a5d5552-19e6-486e-94bf-c6a91b39b987","Type":"ContainerStarted","Data":"43bd56f0ab42cc30c3e261c292e10da0bbf07d0cee87f52aa019de9b58074624"}
Jul 21 16:45:03.184973 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:03.184917    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9" podStartSLOduration=0.722658248 podStartE2EDuration="3.184898629s" totalImagesPullingTime="2.462240381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:45:03.183977303 +0000 UTC m=+998.683338546" watchObservedRunningTime="2026-07-21 16:45:03.184898629 +0000 UTC m=+998.684259876"
Jul 21 16:45:04.171635 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:04.171596    2580 generic.go:356] "Generic (PLEG): container finished" podID="3a5d5552-19e6-486e-94bf-c6a91b39b987" containerID="43bd56f0ab42cc30c3e261c292e10da0bbf07d0cee87f52aa019de9b58074624" exitCode=0
Jul 21 16:45:04.171993 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:04.171682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9" event={"ID":"3a5d5552-19e6-486e-94bf-c6a91b39b987","Type":"ContainerDied","Data":"43bd56f0ab42cc30c3e261c292e10da0bbf07d0cee87f52aa019de9b58074624"}
Jul 21 16:45:05.306281 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:05.306255    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9"
Jul 21 16:45:06.181428 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:06.181398    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9" event={"ID":"3a5d5552-19e6-486e-94bf-c6a91b39b987","Type":"ContainerDied","Data":"2dd71e3686c7336e744eb6b03c460f47743c6ebf23bf53fbee60fd3cf6d228f2"}
Jul 21 16:45:06.181618 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:06.181438    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-qmhj9"
Jul 21 16:45:06.181618 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:45:06.181448    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2dd71e3686c7336e744eb6b03c460f47743c6ebf23bf53fbee60fd3cf6d228f2"
Jul 21 16:50:18.587191 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.587156    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"]
Jul 21 16:50:18.587618 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.587479    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a5d5552-19e6-486e-94bf-c6a91b39b987" containerName="cleanup"
Jul 21 16:50:18.587618 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.587492    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a5d5552-19e6-486e-94bf-c6a91b39b987" containerName="cleanup"
Jul 21 16:50:18.587618 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.587569    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a5d5552-19e6-486e-94bf-c6a91b39b987" containerName="cleanup"
Jul 21 16:50:18.590667 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.590651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.593595 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.593578    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-bd6e2e2a-kserve-self-signed-certs\""
Jul 21 16:50:18.601154 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.601129    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"]
Jul 21 16:50:18.727479 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.727447    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tmp-dir\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.727479 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.727476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tls-certs\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.727770 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.727508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lzjx\" (UniqueName: \"kubernetes.io/projected/38de4968-cc5b-4d4d-a70d-dae0c4afd757-kube-api-access-5lzjx\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.727770 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.727649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-home\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.727770 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.727688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-dshm\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.727770 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.727745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-model-cache\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.828948 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.828913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lzjx\" (UniqueName: \"kubernetes.io/projected/38de4968-cc5b-4d4d-a70d-dae0c4afd757-kube-api-access-5lzjx\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.828948 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.828952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-home\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.829176 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.828970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-dshm\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.829176 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.829094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-model-cache\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.829333 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.829193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tmp-dir\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.829333 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.829211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tls-certs\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.829432 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.829413    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-home\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.829492 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.829418    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-model-cache\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.829578 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.829505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tmp-dir\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.831273 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.831257    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-dshm\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.831815 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.831793    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tls-certs\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.837891 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.837834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lzjx\" (UniqueName: \"kubernetes.io/projected/38de4968-cc5b-4d4d-a70d-dae0c4afd757-kube-api-access-5lzjx\") pod \"e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") " pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:18.899779 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:18.899758    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:19.031087 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:19.031055    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"]
Jul 21 16:50:19.033997 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:50:19.033971    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38de4968_cc5b_4d4d_a70d_dae0c4afd757.slice/crio-4f50337485f9bf5f7dea595b4e468678a9e80b1c0f8acae3b6031ab950c52d66 WatchSource:0}: Error finding container 4f50337485f9bf5f7dea595b4e468678a9e80b1c0f8acae3b6031ab950c52d66: Status 404 returned error can't find the container with id 4f50337485f9bf5f7dea595b4e468678a9e80b1c0f8acae3b6031ab950c52d66
Jul 21 16:50:19.462499 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:19.462466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx" event={"ID":"38de4968-cc5b-4d4d-a70d-dae0c4afd757","Type":"ContainerStarted","Data":"a64294ac0cdd291d0d8d92fcaa6b77de9804ba0fbc453e08c7e2a3874b001cfb"}
Jul 21 16:50:19.462499 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:19.462505    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx" event={"ID":"38de4968-cc5b-4d4d-a70d-dae0c4afd757","Type":"ContainerStarted","Data":"4f50337485f9bf5f7dea595b4e468678a9e80b1c0f8acae3b6031ab950c52d66"}
Jul 21 16:50:19.482742 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:19.482695    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx" podStartSLOduration=1.482680394 podStartE2EDuration="1.482680394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:50:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:50:19.481913213 +0000 UTC m=+1314.981274458" watchObservedRunningTime="2026-07-21 16:50:19.482680394 +0000 UTC m=+1314.982041683"
Jul 21 16:50:28.900453 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:28.900415    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:28.901025 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:28.900481    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:28.913033 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:28.913010    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:29.519823 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:29.519792    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:50:50.488805 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:50:50.488768    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"]
Jul 21 16:51:05.509272 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.509234    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx" podUID="38de4968-cc5b-4d4d-a70d-dae0c4afd757" containerName="main" containerID="cri-o://a64294ac0cdd291d0d8d92fcaa6b77de9804ba0fbc453e08c7e2a3874b001cfb" gracePeriod=45
Jul 21 16:51:05.669039 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.669011    2580 generic.go:356] "Generic (PLEG): container finished" podID="38de4968-cc5b-4d4d-a70d-dae0c4afd757" containerID="a64294ac0cdd291d0d8d92fcaa6b77de9804ba0fbc453e08c7e2a3874b001cfb" exitCode=0
Jul 21 16:51:05.669178 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.669066    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx" event={"ID":"38de4968-cc5b-4d4d-a70d-dae0c4afd757","Type":"ContainerDied","Data":"a64294ac0cdd291d0d8d92fcaa6b77de9804ba0fbc453e08c7e2a3874b001cfb"}
Jul 21 16:51:05.758410 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.758385    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:51:05.775553 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.775494    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lzjx\" (UniqueName: \"kubernetes.io/projected/38de4968-cc5b-4d4d-a70d-dae0c4afd757-kube-api-access-5lzjx\") pod \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") "
Jul 21 16:51:05.775553 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.775541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tls-certs\") pod \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") "
Jul 21 16:51:05.775703 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.775565    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tmp-dir\") pod \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") "
Jul 21 16:51:05.775703 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.775610    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-dshm\") pod \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") "
Jul 21 16:51:05.775703 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.775626    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-model-cache\") pod \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") "
Jul 21 16:51:05.775703 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.775644    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-home\") pod \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\" (UID: \"38de4968-cc5b-4d4d-a70d-dae0c4afd757\") "
Jul 21 16:51:05.775900 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.775864    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "38de4968-cc5b-4d4d-a70d-dae0c4afd757" (UID: "38de4968-cc5b-4d4d-a70d-dae0c4afd757"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 16:51:05.775959 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.775884    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-model-cache" (OuterVolumeSpecName: "model-cache") pod "38de4968-cc5b-4d4d-a70d-dae0c4afd757" (UID: "38de4968-cc5b-4d4d-a70d-dae0c4afd757"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 16:51:05.776016 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.775967    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-home" (OuterVolumeSpecName: "home") pod "38de4968-cc5b-4d4d-a70d-dae0c4afd757" (UID: "38de4968-cc5b-4d4d-a70d-dae0c4afd757"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 16:51:05.777855 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.777831    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-dshm" (OuterVolumeSpecName: "dshm") pod "38de4968-cc5b-4d4d-a70d-dae0c4afd757" (UID: "38de4968-cc5b-4d4d-a70d-dae0c4afd757"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 16:51:05.777943 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.777899    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "38de4968-cc5b-4d4d-a70d-dae0c4afd757" (UID: "38de4968-cc5b-4d4d-a70d-dae0c4afd757"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:51:05.777943 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.777913    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38de4968-cc5b-4d4d-a70d-dae0c4afd757-kube-api-access-5lzjx" (OuterVolumeSpecName: "kube-api-access-5lzjx") pod "38de4968-cc5b-4d4d-a70d-dae0c4afd757" (UID: "38de4968-cc5b-4d4d-a70d-dae0c4afd757"). InnerVolumeSpecName "kube-api-access-5lzjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:51:05.876604 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.876567    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lzjx\" (UniqueName: \"kubernetes.io/projected/38de4968-cc5b-4d4d-a70d-dae0c4afd757-kube-api-access-5lzjx\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:51:05.876604 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.876591    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tls-certs\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:51:05.876604 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.876604    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-tmp-dir\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:51:05.876824 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.876618    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-dshm\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:51:05.876824 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.876629    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-model-cache\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:51:05.876824 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:05.876639    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38de4968-cc5b-4d4d-a70d-dae0c4afd757-home\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:51:06.674132 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:06.674103    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"
Jul 21 16:51:06.674132 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:06.674116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx" event={"ID":"38de4968-cc5b-4d4d-a70d-dae0c4afd757","Type":"ContainerDied","Data":"4f50337485f9bf5f7dea595b4e468678a9e80b1c0f8acae3b6031ab950c52d66"}
Jul 21 16:51:06.674698 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:06.674159    2580 scope.go:117] "RemoveContainer" containerID="a64294ac0cdd291d0d8d92fcaa6b77de9804ba0fbc453e08c7e2a3874b001cfb"
Jul 21 16:51:06.698336 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:06.698315    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"]
Jul 21 16:51:06.702107 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:06.702086    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-bd6e2e2a-kserve-7685b87699-9f5fx"]
Jul 21 16:51:06.939312 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:51:06.939225    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38de4968-cc5b-4d4d-a70d-dae0c4afd757" path="/var/lib/kubelet/pods/38de4968-cc5b-4d4d-a70d-dae0c4afd757/volumes"
Jul 21 16:55:03.787991 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:03.787959    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"]
Jul 21 16:55:03.788399 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:03.788255    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38de4968-cc5b-4d4d-a70d-dae0c4afd757" containerName="main"
Jul 21 16:55:03.788399 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:03.788266    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="38de4968-cc5b-4d4d-a70d-dae0c4afd757" containerName="main"
Jul 21 16:55:03.788399 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:03.788337    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="38de4968-cc5b-4d4d-a70d-dae0c4afd757" containerName="main"
Jul 21 16:55:03.791002 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:03.790977    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"
Jul 21 16:55:03.794260 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:03.794236    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 16:55:03.794370 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:03.794317    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 16:55:03.803778 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:03.803752    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"]
Jul 21 16:55:03.917092 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:03.917062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e169ca-c499-4d4a-8e34-d97a5f089db9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6a668444-s9gf4\" (UID: \"a4e169ca-c499-4d4a-8e34-d97a5f089db9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"
Jul 21 16:55:04.017488 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:04.017443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e169ca-c499-4d4a-8e34-d97a5f089db9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6a668444-s9gf4\" (UID: \"a4e169ca-c499-4d4a-8e34-d97a5f089db9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"
Jul 21 16:55:04.018061 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:04.018041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e169ca-c499-4d4a-8e34-d97a5f089db9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6a668444-s9gf4\" (UID: \"a4e169ca-c499-4d4a-8e34-d97a5f089db9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"
Jul 21 16:55:04.101289 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:04.101217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"
Jul 21 16:55:04.249922 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:04.249890    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"]
Jul 21 16:55:04.253337 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:55:04.253313    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e169ca_c499_4d4a_8e34_d97a5f089db9.slice/crio-7509b0f67c9b4eccf14693e8579109930ccd736a71eb1f8327de82cbb5354702 WatchSource:0}: Error finding container 7509b0f67c9b4eccf14693e8579109930ccd736a71eb1f8327de82cbb5354702: Status 404 returned error can't find the container with id 7509b0f67c9b4eccf14693e8579109930ccd736a71eb1f8327de82cbb5354702
Jul 21 16:55:04.644427 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:04.644398    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4" event={"ID":"a4e169ca-c499-4d4a-8e34-d97a5f089db9","Type":"ContainerStarted","Data":"92d784042b138f6432a8cae49836825d2bbcb083adf76e987b6b080663756645"}
Jul 21 16:55:04.644596 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:04.644451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4" event={"ID":"a4e169ca-c499-4d4a-8e34-d97a5f089db9","Type":"ContainerStarted","Data":"7509b0f67c9b4eccf14693e8579109930ccd736a71eb1f8327de82cbb5354702"}
Jul 21 16:55:04.673170 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:04.673076    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4" podStartSLOduration=1.673062133 podStartE2EDuration="1.673062133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:55:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:55:04.669042232 +0000 UTC m=+1600.168403475" watchObservedRunningTime="2026-07-21 16:55:04.673062133 +0000 UTC m=+1600.172423378"
Jul 21 16:55:05.650923 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:05.650889    2580 generic.go:356] "Generic (PLEG): container finished" podID="a4e169ca-c499-4d4a-8e34-d97a5f089db9" containerID="92d784042b138f6432a8cae49836825d2bbcb083adf76e987b6b080663756645" exitCode=0
Jul 21 16:55:05.651309 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:05.650961    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4" event={"ID":"a4e169ca-c499-4d4a-8e34-d97a5f089db9","Type":"ContainerDied","Data":"92d784042b138f6432a8cae49836825d2bbcb083adf76e987b6b080663756645"}
Jul 21 16:55:06.780576 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:06.780553    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"
Jul 21 16:55:06.943977 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:06.943904    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e169ca-c499-4d4a-8e34-d97a5f089db9-maas-api-service-ca\") pod \"a4e169ca-c499-4d4a-8e34-d97a5f089db9\" (UID: \"a4e169ca-c499-4d4a-8e34-d97a5f089db9\") "
Jul 21 16:55:06.944287 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:06.944252    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4e169ca-c499-4d4a-8e34-d97a5f089db9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4e169ca-c499-4d4a-8e34-d97a5f089db9" (UID: "a4e169ca-c499-4d4a-8e34-d97a5f089db9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:55:07.044961 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:07.044927    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e169ca-c499-4d4a-8e34-d97a5f089db9-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:55:07.661307 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:07.661272    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4" event={"ID":"a4e169ca-c499-4d4a-8e34-d97a5f089db9","Type":"ContainerDied","Data":"7509b0f67c9b4eccf14693e8579109930ccd736a71eb1f8327de82cbb5354702"}
Jul 21 16:55:07.661307 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:07.661303    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"
Jul 21 16:55:07.661549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:07.661321    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7509b0f67c9b4eccf14693e8579109930ccd736a71eb1f8327de82cbb5354702"
Jul 21 16:55:18.914539 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:18.914436    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"]
Jul 21 16:55:18.917714 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:18.917690    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a668444-s9gf4"]
Jul 21 16:55:18.939222 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:18.939191    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4e169ca-c499-4d4a-8e34-d97a5f089db9" path="/var/lib/kubelet/pods/a4e169ca-c499-4d4a-8e34-d97a5f089db9/volumes"
Jul 21 16:55:22.598886 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.598850    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"]
Jul 21 16:55:22.599320 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.599275    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4e169ca-c499-4d4a-8e34-d97a5f089db9" containerName="revoke-keys"
Jul 21 16:55:22.599320 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.599292    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4e169ca-c499-4d4a-8e34-d97a5f089db9" containerName="revoke-keys"
Jul 21 16:55:22.599431 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.599371    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4e169ca-c499-4d4a-8e34-d97a5f089db9" containerName="revoke-keys"
Jul 21 16:55:22.602363 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.602342    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:22.605267 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.605247    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9g9wv\""
Jul 21 16:55:22.605377 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.605287    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-97b8dc80-serving-cert\""
Jul 21 16:55:22.606133 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.606112    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 16:55:22.606211 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.606172    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 16:55:22.621204 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.621187    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"]
Jul 21 16:55:22.661702 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.661673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-maas-api-tls\") pod \"maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:22.661832 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.661724    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-openshift-service-ca\") pod \"maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:22.661832 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.661775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chr4c\" (UniqueName: \"kubernetes.io/projected/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-kube-api-access-chr4c\") pod \"maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:22.762893 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.762861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-maas-api-tls\") pod \"maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:22.763040 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.762912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-openshift-service-ca\") pod \"maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:22.763040 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.762952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chr4c\" (UniqueName: \"kubernetes.io/projected/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-kube-api-access-chr4c\") pod \"maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:22.763040 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:55:22.763023    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-serving-cert: secret "maas-api-e2e-migrate-97b8dc80-serving-cert" not found
Jul 21 16:55:22.763171 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:55:22.763099    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-maas-api-tls podName:8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292 nodeName:}" failed. No retries permitted until 2026-07-21 16:55:23.263078312 +0000 UTC m=+1618.762439538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-maas-api-tls") pod "maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx" (UID: "8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292") : secret "maas-api-e2e-migrate-97b8dc80-serving-cert" not found
Jul 21 16:55:22.763556 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.763539    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-openshift-service-ca\") pod \"maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:22.776220 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:22.776192    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chr4c\" (UniqueName: \"kubernetes.io/projected/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-kube-api-access-chr4c\") pod \"maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:23.266466 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:23.266433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-maas-api-tls\") pod \"maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:23.268942 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:23.268918    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-maas-api-tls\") pod \"maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:23.512457 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:23.512416    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:23.659676 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:23.659641    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"]
Jul 21 16:55:23.663867 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:55:23.663842    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e1116e7_79a8_49e7_b2f4_fa4ed2cdb292.slice/crio-7e446897307440da9cd2cea944f2eb9bbbb2ca87459064308e0a2958a2980c14 WatchSource:0}: Error finding container 7e446897307440da9cd2cea944f2eb9bbbb2ca87459064308e0a2958a2980c14: Status 404 returned error can't find the container with id 7e446897307440da9cd2cea944f2eb9bbbb2ca87459064308e0a2958a2980c14
Jul 21 16:55:23.665390 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:23.665367    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 16:55:23.726761 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:23.726726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx" event={"ID":"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292","Type":"ContainerStarted","Data":"7e446897307440da9cd2cea944f2eb9bbbb2ca87459064308e0a2958a2980c14"}
Jul 21 16:55:26.741295 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:26.741260    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx" event={"ID":"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292","Type":"ContainerStarted","Data":"055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398"}
Jul 21 16:55:26.741703 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:26.741383    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:26.758236 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:26.758191    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx" podStartSLOduration=2.436082013 podStartE2EDuration="4.758178125s" totalImagesPullingTime="2.322096112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:55:26.75733233 +0000 UTC m=+1622.256693575" watchObservedRunningTime="2026-07-21 16:55:26.758178125 +0000 UTC m=+1622.257539412"
Jul 21 16:55:32.750837 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:32.750802    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:47.458702 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:47.458670    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"]
Jul 21 16:55:47.462074 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:47.462056    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"
Jul 21 16:55:47.464711 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:47.464686    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 16:55:47.469185 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:47.469165    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"]
Jul 21 16:55:47.562332 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:47.562301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a474a633-f92d-43ab-907a-de4dc141e4eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf\" (UID: \"a474a633-f92d-43ab-907a-de4dc141e4eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"
Jul 21 16:55:47.663558 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:47.663505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a474a633-f92d-43ab-907a-de4dc141e4eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf\" (UID: \"a474a633-f92d-43ab-907a-de4dc141e4eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"
Jul 21 16:55:47.664112 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:47.664091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a474a633-f92d-43ab-907a-de4dc141e4eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf\" (UID: \"a474a633-f92d-43ab-907a-de4dc141e4eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"
Jul 21 16:55:47.774138 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:47.774079    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"
Jul 21 16:55:47.901941 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:47.901910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"]
Jul 21 16:55:47.904757 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:55:47.904736    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda474a633_f92d_43ab_907a_de4dc141e4eb.slice/crio-7f1ceadb33fce2230f527a34c366dab4b9a1c428ea9ba3b2726aae6ad9f6f9b7 WatchSource:0}: Error finding container 7f1ceadb33fce2230f527a34c366dab4b9a1c428ea9ba3b2726aae6ad9f6f9b7: Status 404 returned error can't find the container with id 7f1ceadb33fce2230f527a34c366dab4b9a1c428ea9ba3b2726aae6ad9f6f9b7
Jul 21 16:55:48.838988 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:48.838956    2580 generic.go:356] "Generic (PLEG): container finished" podID="a474a633-f92d-43ab-907a-de4dc141e4eb" containerID="619592a85c027f4ee0675b514f2fc03cbac634b12e57cd97fb29a11b6f80a16d" exitCode=0
Jul 21 16:55:48.839313 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:48.839005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf" event={"ID":"a474a633-f92d-43ab-907a-de4dc141e4eb","Type":"ContainerDied","Data":"619592a85c027f4ee0675b514f2fc03cbac634b12e57cd97fb29a11b6f80a16d"}
Jul 21 16:55:48.839313 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:48.839054    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf" event={"ID":"a474a633-f92d-43ab-907a-de4dc141e4eb","Type":"ContainerStarted","Data":"7f1ceadb33fce2230f527a34c366dab4b9a1c428ea9ba3b2726aae6ad9f6f9b7"}
Jul 21 16:55:49.967913 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:49.967890    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"
Jul 21 16:55:50.082234 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:50.082204    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a474a633-f92d-43ab-907a-de4dc141e4eb-maas-api-service-ca\") pod \"a474a633-f92d-43ab-907a-de4dc141e4eb\" (UID: \"a474a633-f92d-43ab-907a-de4dc141e4eb\") "
Jul 21 16:55:50.082651 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:50.082616    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a474a633-f92d-43ab-907a-de4dc141e4eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a474a633-f92d-43ab-907a-de4dc141e4eb" (UID: "a474a633-f92d-43ab-907a-de4dc141e4eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:55:50.183781 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:50.183743    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a474a633-f92d-43ab-907a-de4dc141e4eb-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:55:50.848311 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:50.848272    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf" event={"ID":"a474a633-f92d-43ab-907a-de4dc141e4eb","Type":"ContainerDied","Data":"7f1ceadb33fce2230f527a34c366dab4b9a1c428ea9ba3b2726aae6ad9f6f9b7"}
Jul 21 16:55:50.848311 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:50.848296    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"
Jul 21 16:55:50.848502 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:50.848332    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f1ceadb33fce2230f527a34c366dab4b9a1c428ea9ba3b2726aae6ad9f6f9b7"
Jul 21 16:55:57.839163 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:57.839127    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"]
Jul 21 16:55:57.839620 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:57.839374    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx" podUID="8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292" containerName="maas-api" containerID="cri-o://055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398" gracePeriod=30
Jul 21 16:55:58.084333 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.084309    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:58.244749 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.244723    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chr4c\" (UniqueName: \"kubernetes.io/projected/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-kube-api-access-chr4c\") pod \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") "
Jul 21 16:55:58.244884 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.244769    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-maas-api-tls\") pod \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") "
Jul 21 16:55:58.244884 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.244808    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-openshift-service-ca\") pod \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\" (UID: \"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292\") "
Jul 21 16:55:58.245211 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.245186    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292" (UID: "8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:55:58.247129 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.247108    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292" (UID: "8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:55:58.247129 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.247123    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-kube-api-access-chr4c" (OuterVolumeSpecName: "kube-api-access-chr4c") pod "8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292" (UID: "8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292"). InnerVolumeSpecName "kube-api-access-chr4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:55:58.345574 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.345552    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:55:58.345574 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.345574    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chr4c\" (UniqueName: \"kubernetes.io/projected/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-kube-api-access-chr4c\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:55:58.345689 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.345584    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:55:58.885090 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.885053    2580 generic.go:356] "Generic (PLEG): container finished" podID="8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292" containerID="055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398" exitCode=0
Jul 21 16:55:58.885485 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.885103    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"
Jul 21 16:55:58.885485 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.885133    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx" event={"ID":"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292","Type":"ContainerDied","Data":"055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398"}
Jul 21 16:55:58.885485 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.885178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx" event={"ID":"8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292","Type":"ContainerDied","Data":"7e446897307440da9cd2cea944f2eb9bbbb2ca87459064308e0a2958a2980c14"}
Jul 21 16:55:58.885485 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.885199    2580 scope.go:117] "RemoveContainer" containerID="055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398"
Jul 21 16:55:58.895643 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.895581    2580 scope.go:117] "RemoveContainer" containerID="055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398"
Jul 21 16:55:58.895904 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:55:58.895879    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398\": container with ID starting with 055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398 not found: ID does not exist" containerID="055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398"
Jul 21 16:55:58.896006 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.895909    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398"} err="failed to get container status \"055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398\": rpc error: code = NotFound desc = could not find container \"055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398\": container with ID starting with 055a956be4e05d34f133156ff33c75b7105f4db034dcb2a5af787eb706023398 not found: ID does not exist"
Jul 21 16:55:58.908949 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.908926    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"]
Jul 21 16:55:58.915151 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.915133    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-97b8dc80-76c688d685-vk2mx"]
Jul 21 16:55:58.938731 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:55:58.938710    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292" path="/var/lib/kubelet/pods/8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292/volumes"
Jul 21 16:56:02.566681 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:02.566637    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"]
Jul 21 16:56:02.569205 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:02.569176    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-97b8dc80-9zjpf"]
Jul 21 16:56:02.939124 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:02.939093    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a474a633-f92d-43ab-907a-de4dc141e4eb" path="/var/lib/kubelet/pods/a474a633-f92d-43ab-907a-de4dc141e4eb/volumes"
Jul 21 16:56:05.717280 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.717248    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"]
Jul 21 16:56:05.717741 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.717553    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292" containerName="maas-api"
Jul 21 16:56:05.717741 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.717565    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292" containerName="maas-api"
Jul 21 16:56:05.717741 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.717583    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a474a633-f92d-43ab-907a-de4dc141e4eb" containerName="revoke-keys"
Jul 21 16:56:05.717741 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.717588    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a474a633-f92d-43ab-907a-de4dc141e4eb" containerName="revoke-keys"
Jul 21 16:56:05.717741 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.717636    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a474a633-f92d-43ab-907a-de4dc141e4eb" containerName="revoke-keys"
Jul 21 16:56:05.717741 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.717645    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e1116e7-79a8-49e7-b2f4-fa4ed2cdb292" containerName="maas-api"
Jul 21 16:56:05.723931 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.723914    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:05.726474 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.726445    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-26d0f365-serving-cert\""
Jul 21 16:56:05.726615 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.726557    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 16:56:05.727482 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.727464    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9g9wv\""
Jul 21 16:56:05.727617 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.727532    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 16:56:05.737051 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.737029    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"]
Jul 21 16:56:05.805553 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.805509    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-maas-api-tls\") pod \"maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:05.805660 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.805570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-openshift-service-ca\") pod \"maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:05.805660 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.805598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-595nr\" (UniqueName: \"kubernetes.io/projected/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-kube-api-access-595nr\") pod \"maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:05.907037 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.907009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-maas-api-tls\") pod \"maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:05.907181 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.907046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-openshift-service-ca\") pod \"maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:05.907181 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.907071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-595nr\" (UniqueName: \"kubernetes.io/projected/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-kube-api-access-595nr\") pod \"maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:05.907181 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:56:05.907148    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-serving-cert: secret "maas-api-e2e-ait-26d0f365-serving-cert" not found
Jul 21 16:56:05.907336 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:56:05.907223    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-maas-api-tls podName:dd2d76a7-9606-4d4c-b754-abe65bd9eef5 nodeName:}" failed. No retries permitted until 2026-07-21 16:56:06.407205526 +0000 UTC m=+1661.906566756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-maas-api-tls") pod "maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f" (UID: "dd2d76a7-9606-4d4c-b754-abe65bd9eef5") : secret "maas-api-e2e-ait-26d0f365-serving-cert" not found
Jul 21 16:56:05.907639 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.907622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-openshift-service-ca\") pod \"maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:05.916135 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:05.916112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-595nr\" (UniqueName: \"kubernetes.io/projected/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-kube-api-access-595nr\") pod \"maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:06.410776 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:06.410747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-maas-api-tls\") pod \"maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:06.413241 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:06.413213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-maas-api-tls\") pod \"maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:06.633508 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:06.633478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:06.767000 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:06.766972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"]
Jul 21 16:56:06.769535 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:56:06.769497    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd2d76a7_9606_4d4c_b754_abe65bd9eef5.slice/crio-bf0df7e3750604c73f9d4154591e2008210889ad1187066112f5e4ff339d0279 WatchSource:0}: Error finding container bf0df7e3750604c73f9d4154591e2008210889ad1187066112f5e4ff339d0279: Status 404 returned error can't find the container with id bf0df7e3750604c73f9d4154591e2008210889ad1187066112f5e4ff339d0279
Jul 21 16:56:06.922211 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:06.922174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f" event={"ID":"dd2d76a7-9606-4d4c-b754-abe65bd9eef5","Type":"ContainerStarted","Data":"bf0df7e3750604c73f9d4154591e2008210889ad1187066112f5e4ff339d0279"}
Jul 21 16:56:08.932966 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:08.932930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f" event={"ID":"dd2d76a7-9606-4d4c-b754-abe65bd9eef5","Type":"ContainerStarted","Data":"a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c"}
Jul 21 16:56:08.933360 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:08.933048    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:08.952028 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:08.951980    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f" podStartSLOduration=2.674113106 podStartE2EDuration="3.951962921s" totalImagesPullingTime="1.277849815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:56:08.951102037 +0000 UTC m=+1664.450463271" watchObservedRunningTime="2026-07-21 16:56:08.951962921 +0000 UTC m=+1664.451324166"
Jul 21 16:56:14.942429 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:14.942398    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:31.322844 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:31.322811    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"]
Jul 21 16:56:31.327150 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:31.327135    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"
Jul 21 16:56:31.329531 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:31.329491    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 16:56:31.333267 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:31.333242    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"]
Jul 21 16:56:31.413535 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:31.413491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b21c3d55-fd26-48bb-870d-724f593ee3b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-26d0f365-txscf\" (UID: \"b21c3d55-fd26-48bb-870d-724f593ee3b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"
Jul 21 16:56:31.514320 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:31.514284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b21c3d55-fd26-48bb-870d-724f593ee3b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-26d0f365-txscf\" (UID: \"b21c3d55-fd26-48bb-870d-724f593ee3b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"
Jul 21 16:56:31.514900 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:31.514881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b21c3d55-fd26-48bb-870d-724f593ee3b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-26d0f365-txscf\" (UID: \"b21c3d55-fd26-48bb-870d-724f593ee3b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"
Jul 21 16:56:31.638631 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:31.638562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"
Jul 21 16:56:31.972722 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:31.972689    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"]
Jul 21 16:56:31.975023 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:56:31.974988    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb21c3d55_fd26_48bb_870d_724f593ee3b7.slice/crio-7eb7acb0037acbab6d6840ac867ff4b9394dd77461b04cc0af8f90012a1c44e6 WatchSource:0}: Error finding container 7eb7acb0037acbab6d6840ac867ff4b9394dd77461b04cc0af8f90012a1c44e6: Status 404 returned error can't find the container with id 7eb7acb0037acbab6d6840ac867ff4b9394dd77461b04cc0af8f90012a1c44e6
Jul 21 16:56:32.037214 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:32.037179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf" event={"ID":"b21c3d55-fd26-48bb-870d-724f593ee3b7","Type":"ContainerStarted","Data":"7eb7acb0037acbab6d6840ac867ff4b9394dd77461b04cc0af8f90012a1c44e6"}
Jul 21 16:56:33.043314 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:33.043224    2580 generic.go:356] "Generic (PLEG): container finished" podID="b21c3d55-fd26-48bb-870d-724f593ee3b7" containerID="375d5dd5b5c04619444a5e0ff48a4ef4fd09541e7bdf20dc8e737708368c71bf" exitCode=0
Jul 21 16:56:33.043718 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:33.043307    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf" event={"ID":"b21c3d55-fd26-48bb-870d-724f593ee3b7","Type":"ContainerDied","Data":"375d5dd5b5c04619444a5e0ff48a4ef4fd09541e7bdf20dc8e737708368c71bf"}
Jul 21 16:56:34.172200 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:34.172178    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"
Jul 21 16:56:34.236219 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:34.236194    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b21c3d55-fd26-48bb-870d-724f593ee3b7-maas-api-service-ca\") pod \"b21c3d55-fd26-48bb-870d-724f593ee3b7\" (UID: \"b21c3d55-fd26-48bb-870d-724f593ee3b7\") "
Jul 21 16:56:34.236572 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:34.236546    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b21c3d55-fd26-48bb-870d-724f593ee3b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b21c3d55-fd26-48bb-870d-724f593ee3b7" (UID: "b21c3d55-fd26-48bb-870d-724f593ee3b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:56:34.337548 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:34.337466    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b21c3d55-fd26-48bb-870d-724f593ee3b7-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:56:35.057794 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:35.057762    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"
Jul 21 16:56:35.057964 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:35.057792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf" event={"ID":"b21c3d55-fd26-48bb-870d-724f593ee3b7","Type":"ContainerDied","Data":"7eb7acb0037acbab6d6840ac867ff4b9394dd77461b04cc0af8f90012a1c44e6"}
Jul 21 16:56:35.057964 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:35.057883    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7eb7acb0037acbab6d6840ac867ff4b9394dd77461b04cc0af8f90012a1c44e6"
Jul 21 16:56:41.757118 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:41.757090    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"]
Jul 21 16:56:41.757605 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:41.757307    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f" podUID="dd2d76a7-9606-4d4c-b754-abe65bd9eef5" containerName="maas-api" containerID="cri-o://a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c" gracePeriod=30
Jul 21 16:56:42.007286 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.007238    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:42.090069 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.090033    2580 generic.go:356] "Generic (PLEG): container finished" podID="dd2d76a7-9606-4d4c-b754-abe65bd9eef5" containerID="a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c" exitCode=0
Jul 21 16:56:42.090220 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.090082    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"
Jul 21 16:56:42.090220 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.090101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f" event={"ID":"dd2d76a7-9606-4d4c-b754-abe65bd9eef5","Type":"ContainerDied","Data":"a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c"}
Jul 21 16:56:42.090220 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.090153    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f" event={"ID":"dd2d76a7-9606-4d4c-b754-abe65bd9eef5","Type":"ContainerDied","Data":"bf0df7e3750604c73f9d4154591e2008210889ad1187066112f5e4ff339d0279"}
Jul 21 16:56:42.090220 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.090195    2580 scope.go:117] "RemoveContainer" containerID="a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c"
Jul 21 16:56:42.097001 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.096976    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-openshift-service-ca\") pod \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") "
Jul 21 16:56:42.097136 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.097012    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-maas-api-tls\") pod \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") "
Jul 21 16:56:42.097136 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.097076    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-595nr\" (UniqueName: \"kubernetes.io/projected/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-kube-api-access-595nr\") pod \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\" (UID: \"dd2d76a7-9606-4d4c-b754-abe65bd9eef5\") "
Jul 21 16:56:42.097600 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.097551    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dd2d76a7-9606-4d4c-b754-abe65bd9eef5" (UID: "dd2d76a7-9606-4d4c-b754-abe65bd9eef5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:56:42.100008 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.099984    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dd2d76a7-9606-4d4c-b754-abe65bd9eef5" (UID: "dd2d76a7-9606-4d4c-b754-abe65bd9eef5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:56:42.100118 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.100010    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-kube-api-access-595nr" (OuterVolumeSpecName: "kube-api-access-595nr") pod "dd2d76a7-9606-4d4c-b754-abe65bd9eef5" (UID: "dd2d76a7-9606-4d4c-b754-abe65bd9eef5"). InnerVolumeSpecName "kube-api-access-595nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:56:42.110765 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.110732    2580 scope.go:117] "RemoveContainer" containerID="a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c"
Jul 21 16:56:42.111079 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:56:42.111058    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c\": container with ID starting with a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c not found: ID does not exist" containerID="a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c"
Jul 21 16:56:42.111141 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.111087    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c"} err="failed to get container status \"a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c\": rpc error: code = NotFound desc = could not find container \"a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c\": container with ID starting with a3a7a711e68114476b84d5118afb5791012a8d33631cee03b0e219d2acdd6b3c not found: ID does not exist"
Jul 21 16:56:42.198722 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.198696    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-595nr\" (UniqueName: \"kubernetes.io/projected/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-kube-api-access-595nr\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:56:42.198722 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.198718    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:56:42.198871 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.198728    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd2d76a7-9606-4d4c-b754-abe65bd9eef5-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:56:42.413284 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.413257    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"]
Jul 21 16:56:42.414579 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.414558    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26d0f365-7fc44cdc9c-8f26f"]
Jul 21 16:56:42.939409 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:42.939366    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd2d76a7-9606-4d4c-b754-abe65bd9eef5" path="/var/lib/kubelet/pods/dd2d76a7-9606-4d4c-b754-abe65bd9eef5/volumes"
Jul 21 16:56:46.400153 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:46.400075    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"]
Jul 21 16:56:46.403317 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:46.403294    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-26d0f365-txscf"]
Jul 21 16:56:46.938925 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:46.938892    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b21c3d55-fd26-48bb-870d-724f593ee3b7" path="/var/lib/kubelet/pods/b21c3d55-fd26-48bb-870d-724f593ee3b7/volumes"
Jul 21 16:56:51.023380 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.023341    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"]
Jul 21 16:56:51.023932 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.023812    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b21c3d55-fd26-48bb-870d-724f593ee3b7" containerName="revoke-keys"
Jul 21 16:56:51.023932 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.023830    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b21c3d55-fd26-48bb-870d-724f593ee3b7" containerName="revoke-keys"
Jul 21 16:56:51.023932 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.023857    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd2d76a7-9606-4d4c-b754-abe65bd9eef5" containerName="maas-api"
Jul 21 16:56:51.023932 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.023866    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd2d76a7-9606-4d4c-b754-abe65bd9eef5" containerName="maas-api"
Jul 21 16:56:51.024141 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.023951    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b21c3d55-fd26-48bb-870d-724f593ee3b7" containerName="revoke-keys"
Jul 21 16:56:51.024141 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.023963    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd2d76a7-9606-4d4c-b754-abe65bd9eef5" containerName="maas-api"
Jul 21 16:56:51.027898 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.027879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.030545 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.030507    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 16:56:51.030545 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.030536    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-ed714b0f-serving-cert\""
Jul 21 16:56:51.030710 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.030594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9g9wv\""
Jul 21 16:56:51.031487 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.031473    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 16:56:51.041585 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.041565    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"]
Jul 21 16:56:51.067883 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.067858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsbrs\" (UniqueName: \"kubernetes.io/projected/5769a462-f508-4ccc-89bc-6375c4884ab1-kube-api-access-zsbrs\") pod \"maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.067975 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.067900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5769a462-f508-4ccc-89bc-6375c4884ab1-openshift-service-ca\") pod \"maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.067975 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.067925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5769a462-f508-4ccc-89bc-6375c4884ab1-maas-api-tls\") pod \"maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.169303 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.169266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsbrs\" (UniqueName: \"kubernetes.io/projected/5769a462-f508-4ccc-89bc-6375c4884ab1-kube-api-access-zsbrs\") pod \"maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.169486 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.169326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5769a462-f508-4ccc-89bc-6375c4884ab1-openshift-service-ca\") pod \"maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.169486 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.169358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5769a462-f508-4ccc-89bc-6375c4884ab1-maas-api-tls\") pod \"maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.169641 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:56:51.169500    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-serving-cert: secret "maas-api-e2e-derive-ed714b0f-serving-cert" not found
Jul 21 16:56:51.169641 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:56:51.169598    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5769a462-f508-4ccc-89bc-6375c4884ab1-maas-api-tls podName:5769a462-f508-4ccc-89bc-6375c4884ab1 nodeName:}" failed. No retries permitted until 2026-07-21 16:56:51.669576645 +0000 UTC m=+1707.168937867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5769a462-f508-4ccc-89bc-6375c4884ab1-maas-api-tls") pod "maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw" (UID: "5769a462-f508-4ccc-89bc-6375c4884ab1") : secret "maas-api-e2e-derive-ed714b0f-serving-cert" not found
Jul 21 16:56:51.170067 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.170047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5769a462-f508-4ccc-89bc-6375c4884ab1-openshift-service-ca\") pod \"maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.189563 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.189539    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsbrs\" (UniqueName: \"kubernetes.io/projected/5769a462-f508-4ccc-89bc-6375c4884ab1-kube-api-access-zsbrs\") pod \"maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.673992 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.673958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5769a462-f508-4ccc-89bc-6375c4884ab1-maas-api-tls\") pod \"maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.676410 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.676383    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5769a462-f508-4ccc-89bc-6375c4884ab1-maas-api-tls\") pod \"maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:51.938435 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:51.938344    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:52.073399 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:52.073376    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"]
Jul 21 16:56:52.076156 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:56:52.076126    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5769a462_f508_4ccc_89bc_6375c4884ab1.slice/crio-6565b5b2e6d690faf5f06a4e64204e4e464b55d8c3fd418bb55a5b94386e198d WatchSource:0}: Error finding container 6565b5b2e6d690faf5f06a4e64204e4e464b55d8c3fd418bb55a5b94386e198d: Status 404 returned error can't find the container with id 6565b5b2e6d690faf5f06a4e64204e4e464b55d8c3fd418bb55a5b94386e198d
Jul 21 16:56:52.136975 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:52.136944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw" event={"ID":"5769a462-f508-4ccc-89bc-6375c4884ab1","Type":"ContainerStarted","Data":"6565b5b2e6d690faf5f06a4e64204e4e464b55d8c3fd418bb55a5b94386e198d"}
Jul 21 16:56:53.143193 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:53.143160    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw" event={"ID":"5769a462-f508-4ccc-89bc-6375c4884ab1","Type":"ContainerStarted","Data":"a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c"}
Jul 21 16:56:53.143570 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:53.143276    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:56:53.162097 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:53.162051    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw" podStartSLOduration=2.669686331 podStartE2EDuration="3.162037768s" totalImagesPullingTime="492.351437ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:56:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:56:53.159948799 +0000 UTC m=+1708.659310044" watchObservedRunningTime="2026-07-21 16:56:53.162037768 +0000 UTC m=+1708.661399011"
Jul 21 16:56:59.153245 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:56:59.153219    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:57:15.746257 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:15.746223    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"]
Jul 21 16:57:15.749661 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:15.749643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"
Jul 21 16:57:15.753478 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:15.752634    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 16:57:15.761423 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:15.761403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"]
Jul 21 16:57:15.865662 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:15.865634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76f84594-d624-46f1-be9d-1644ee457661-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q\" (UID: \"76f84594-d624-46f1-be9d-1644ee457661\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"
Jul 21 16:57:15.966463 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:15.966435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76f84594-d624-46f1-be9d-1644ee457661-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q\" (UID: \"76f84594-d624-46f1-be9d-1644ee457661\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"
Jul 21 16:57:15.967051 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:15.967028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76f84594-d624-46f1-be9d-1644ee457661-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q\" (UID: \"76f84594-d624-46f1-be9d-1644ee457661\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"
Jul 21 16:57:16.066415 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:16.066364    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"
Jul 21 16:57:16.196481 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:16.196458    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"]
Jul 21 16:57:16.198863 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:57:16.198838    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f84594_d624_46f1_be9d_1644ee457661.slice/crio-4b4aeb30fcf8989f564a3dee60cf1cdc50b15ccb5f4a0e2a3e3e925830ca31a3 WatchSource:0}: Error finding container 4b4aeb30fcf8989f564a3dee60cf1cdc50b15ccb5f4a0e2a3e3e925830ca31a3: Status 404 returned error can't find the container with id 4b4aeb30fcf8989f564a3dee60cf1cdc50b15ccb5f4a0e2a3e3e925830ca31a3
Jul 21 16:57:16.244805 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:16.244781    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q" event={"ID":"76f84594-d624-46f1-be9d-1644ee457661","Type":"ContainerStarted","Data":"4b4aeb30fcf8989f564a3dee60cf1cdc50b15ccb5f4a0e2a3e3e925830ca31a3"}
Jul 21 16:57:17.249997 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:17.249959    2580 generic.go:356] "Generic (PLEG): container finished" podID="76f84594-d624-46f1-be9d-1644ee457661" containerID="65a8cb11e66c710683a42f01f332145ffb92c0aea285028cd6f662dc81b5c330" exitCode=0
Jul 21 16:57:17.250359 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:17.250005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q" event={"ID":"76f84594-d624-46f1-be9d-1644ee457661","Type":"ContainerDied","Data":"65a8cb11e66c710683a42f01f332145ffb92c0aea285028cd6f662dc81b5c330"}
Jul 21 16:57:18.375384 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:18.375362    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"
Jul 21 16:57:18.485103 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:18.485080    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76f84594-d624-46f1-be9d-1644ee457661-maas-api-service-ca\") pod \"76f84594-d624-46f1-be9d-1644ee457661\" (UID: \"76f84594-d624-46f1-be9d-1644ee457661\") "
Jul 21 16:57:18.485394 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:18.485371    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f84594-d624-46f1-be9d-1644ee457661-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "76f84594-d624-46f1-be9d-1644ee457661" (UID: "76f84594-d624-46f1-be9d-1644ee457661"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:57:18.586019 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:18.585948    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76f84594-d624-46f1-be9d-1644ee457661-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:57:18.969474 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:57:18.969449    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f84594_d624_46f1_be9d_1644ee457661.slice\": RecentStats: unable to find data in memory cache]"
Jul 21 16:57:19.259763 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:19.259695    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"
Jul 21 16:57:19.259763 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:19.259694    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q" event={"ID":"76f84594-d624-46f1-be9d-1644ee457661","Type":"ContainerDied","Data":"4b4aeb30fcf8989f564a3dee60cf1cdc50b15ccb5f4a0e2a3e3e925830ca31a3"}
Jul 21 16:57:19.259763 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:19.259745    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b4aeb30fcf8989f564a3dee60cf1cdc50b15ccb5f4a0e2a3e3e925830ca31a3"
Jul 21 16:57:26.465507 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.465472    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"]
Jul 21 16:57:26.465986 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.465801    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw" podUID="5769a462-f508-4ccc-89bc-6375c4884ab1" containerName="maas-api" containerID="cri-o://a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c" gracePeriod=30
Jul 21 16:57:26.701597 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.701575    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:57:26.845886 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.845810    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5769a462-f508-4ccc-89bc-6375c4884ab1-openshift-service-ca\") pod \"5769a462-f508-4ccc-89bc-6375c4884ab1\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") "
Jul 21 16:57:26.845886 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.845872    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5769a462-f508-4ccc-89bc-6375c4884ab1-maas-api-tls\") pod \"5769a462-f508-4ccc-89bc-6375c4884ab1\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") "
Jul 21 16:57:26.846092 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.845950    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsbrs\" (UniqueName: \"kubernetes.io/projected/5769a462-f508-4ccc-89bc-6375c4884ab1-kube-api-access-zsbrs\") pod \"5769a462-f508-4ccc-89bc-6375c4884ab1\" (UID: \"5769a462-f508-4ccc-89bc-6375c4884ab1\") "
Jul 21 16:57:26.846172 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.846147    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5769a462-f508-4ccc-89bc-6375c4884ab1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5769a462-f508-4ccc-89bc-6375c4884ab1" (UID: "5769a462-f508-4ccc-89bc-6375c4884ab1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:57:26.848196 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.848166    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5769a462-f508-4ccc-89bc-6375c4884ab1-kube-api-access-zsbrs" (OuterVolumeSpecName: "kube-api-access-zsbrs") pod "5769a462-f508-4ccc-89bc-6375c4884ab1" (UID: "5769a462-f508-4ccc-89bc-6375c4884ab1"). InnerVolumeSpecName "kube-api-access-zsbrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:57:26.848309 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.848200    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5769a462-f508-4ccc-89bc-6375c4884ab1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5769a462-f508-4ccc-89bc-6375c4884ab1" (UID: "5769a462-f508-4ccc-89bc-6375c4884ab1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:57:26.946717 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.946692    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5769a462-f508-4ccc-89bc-6375c4884ab1-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:57:26.946717 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.946717    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5769a462-f508-4ccc-89bc-6375c4884ab1-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:57:26.946901 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:26.946727    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsbrs\" (UniqueName: \"kubernetes.io/projected/5769a462-f508-4ccc-89bc-6375c4884ab1-kube-api-access-zsbrs\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:57:27.298680 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:27.298641    2580 generic.go:356] "Generic (PLEG): container finished" podID="5769a462-f508-4ccc-89bc-6375c4884ab1" containerID="a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c" exitCode=0
Jul 21 16:57:27.298860 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:27.298684    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"
Jul 21 16:57:27.298860 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:27.298725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw" event={"ID":"5769a462-f508-4ccc-89bc-6375c4884ab1","Type":"ContainerDied","Data":"a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c"}
Jul 21 16:57:27.298860 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:27.298792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw" event={"ID":"5769a462-f508-4ccc-89bc-6375c4884ab1","Type":"ContainerDied","Data":"6565b5b2e6d690faf5f06a4e64204e4e464b55d8c3fd418bb55a5b94386e198d"}
Jul 21 16:57:27.298860 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:27.298817    2580 scope.go:117] "RemoveContainer" containerID="a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c"
Jul 21 16:57:27.307001 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:27.306984    2580 scope.go:117] "RemoveContainer" containerID="a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c"
Jul 21 16:57:27.307262 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:57:27.307246    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c\": container with ID starting with a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c not found: ID does not exist" containerID="a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c"
Jul 21 16:57:27.307321 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:27.307271    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c"} err="failed to get container status \"a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c\": rpc error: code = NotFound desc = could not find container \"a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c\": container with ID starting with a36080b0c649680c79659b15403cc7104870eff4a8621826190f1c6f4b518e1c not found: ID does not exist"
Jul 21 16:57:27.318503 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:27.318483    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"]
Jul 21 16:57:27.322595 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:27.322576    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ed714b0f-7f9bdc5d5c-qx4tw"]
Jul 21 16:57:28.940333 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:28.940297    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5769a462-f508-4ccc-89bc-6375c4884ab1" path="/var/lib/kubelet/pods/5769a462-f508-4ccc-89bc-6375c4884ab1/volumes"
Jul 21 16:57:30.844424 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:30.844395    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"]
Jul 21 16:57:30.846814 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:30.846786    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ed714b0f-sh55q"]
Jul 21 16:57:30.939309 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:57:30.939269    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f84594-d624-46f1-be9d-1644ee457661" path="/var/lib/kubelet/pods/76f84594-d624-46f1-be9d-1644ee457661/volumes"
Jul 21 16:59:21.353214 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.353179    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"]
Jul 21 16:59:21.353705 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.353479    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f84594-d624-46f1-be9d-1644ee457661" containerName="revoke-keys"
Jul 21 16:59:21.353705 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.353490    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f84594-d624-46f1-be9d-1644ee457661" containerName="revoke-keys"
Jul 21 16:59:21.353705 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.353528    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5769a462-f508-4ccc-89bc-6375c4884ab1" containerName="maas-api"
Jul 21 16:59:21.353705 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.353533    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5769a462-f508-4ccc-89bc-6375c4884ab1" containerName="maas-api"
Jul 21 16:59:21.353705 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.353584    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5769a462-f508-4ccc-89bc-6375c4884ab1" containerName="maas-api"
Jul 21 16:59:21.353705 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.353592    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f84594-d624-46f1-be9d-1644ee457661" containerName="revoke-keys"
Jul 21 16:59:21.362220 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.362192    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:21.364681 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.364659    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-13ffe089-serving-cert\""
Jul 21 16:59:21.368683 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.368654    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 16:59:21.369434 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.369036    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 16:59:21.369434 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.369316    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"]
Jul 21 16:59:21.370370 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.370351    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9g9wv\""
Jul 21 16:59:21.412249 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.412212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxww8\" (UniqueName: \"kubernetes.io/projected/baa8032b-613a-493c-8c6e-a75f2ef35b33-kube-api-access-lxww8\") pod \"maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:21.412443 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.412254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa8032b-613a-493c-8c6e-a75f2ef35b33-maas-api-tls\") pod \"maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:21.412443 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.412279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa8032b-613a-493c-8c6e-a75f2ef35b33-openshift-service-ca\") pod \"maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:21.512796 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.512755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxww8\" (UniqueName: \"kubernetes.io/projected/baa8032b-613a-493c-8c6e-a75f2ef35b33-kube-api-access-lxww8\") pod \"maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:21.512796 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.512798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa8032b-613a-493c-8c6e-a75f2ef35b33-maas-api-tls\") pod \"maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:21.513062 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.512819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa8032b-613a-493c-8c6e-a75f2ef35b33-openshift-service-ca\") pod \"maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:21.513062 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:59:21.512925    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-serving-cert: secret "maas-api-e2e-mt-13ffe089-serving-cert" not found
Jul 21 16:59:21.513062 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:59:21.513000    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baa8032b-613a-493c-8c6e-a75f2ef35b33-maas-api-tls podName:baa8032b-613a-493c-8c6e-a75f2ef35b33 nodeName:}" failed. No retries permitted until 2026-07-21 16:59:22.012984745 +0000 UTC m=+1857.512345967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/baa8032b-613a-493c-8c6e-a75f2ef35b33-maas-api-tls") pod "maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl" (UID: "baa8032b-613a-493c-8c6e-a75f2ef35b33") : secret "maas-api-e2e-mt-13ffe089-serving-cert" not found
Jul 21 16:59:21.513506 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.513485    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa8032b-613a-493c-8c6e-a75f2ef35b33-openshift-service-ca\") pod \"maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:21.529170 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:21.529136    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxww8\" (UniqueName: \"kubernetes.io/projected/baa8032b-613a-493c-8c6e-a75f2ef35b33-kube-api-access-lxww8\") pod \"maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:22.017272 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:22.017230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa8032b-613a-493c-8c6e-a75f2ef35b33-maas-api-tls\") pod \"maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:22.019879 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:22.019856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa8032b-613a-493c-8c6e-a75f2ef35b33-maas-api-tls\") pod \"maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:22.275035 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:22.274936    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:22.418028 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:22.417945    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"]
Jul 21 16:59:22.421835 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:59:22.421804    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaa8032b_613a_493c_8c6e_a75f2ef35b33.slice/crio-4e29d6216c014f201f23a410e5f3b4d6d3fa5d479c5fd4d9a0c4b0f74a9f5bbf WatchSource:0}: Error finding container 4e29d6216c014f201f23a410e5f3b4d6d3fa5d479c5fd4d9a0c4b0f74a9f5bbf: Status 404 returned error can't find the container with id 4e29d6216c014f201f23a410e5f3b4d6d3fa5d479c5fd4d9a0c4b0f74a9f5bbf
Jul 21 16:59:22.812008 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:22.811965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl" event={"ID":"baa8032b-613a-493c-8c6e-a75f2ef35b33","Type":"ContainerStarted","Data":"4e29d6216c014f201f23a410e5f3b4d6d3fa5d479c5fd4d9a0c4b0f74a9f5bbf"}
Jul 21 16:59:23.817856 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:23.817818    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl" event={"ID":"baa8032b-613a-493c-8c6e-a75f2ef35b33","Type":"ContainerStarted","Data":"3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c"}
Jul 21 16:59:23.818254 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:23.817886    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:23.834615 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:23.834555    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl" podStartSLOduration=2.365348075 podStartE2EDuration="2.834506005s" totalImagesPullingTime="469.15793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:59:23.834412593 +0000 UTC m=+1859.333773874" watchObservedRunningTime="2026-07-21 16:59:23.834506005 +0000 UTC m=+1859.333867249"
Jul 21 16:59:29.828283 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:29.828251    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:32.931278 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:32.931242    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"]
Jul 21 16:59:32.934723 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:32.934699    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"
Jul 21 16:59:32.937283 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:32.937253    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 16:59:32.942843 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:32.942371    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"]
Jul 21 16:59:33.016150 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:33.016101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1a8365a-d37e-4e71-93cd-2b4d75c2544b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc\" (UID: \"f1a8365a-d37e-4e71-93cd-2b4d75c2544b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"
Jul 21 16:59:33.117201 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:33.117161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1a8365a-d37e-4e71-93cd-2b4d75c2544b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc\" (UID: \"f1a8365a-d37e-4e71-93cd-2b4d75c2544b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"
Jul 21 16:59:33.118016 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:33.117997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1a8365a-d37e-4e71-93cd-2b4d75c2544b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc\" (UID: \"f1a8365a-d37e-4e71-93cd-2b4d75c2544b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"
Jul 21 16:59:33.248003 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:33.247917    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"
Jul 21 16:59:33.384636 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:33.384599    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"]
Jul 21 16:59:33.388612 ip-10-0-141-158 kubenswrapper[2580]: W0721 16:59:33.388566    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a8365a_d37e_4e71_93cd_2b4d75c2544b.slice/crio-7cc5b7b497dd6b2e9c7a3eeb9c365a00910b02ef062065004cf1c3f594882992 WatchSource:0}: Error finding container 7cc5b7b497dd6b2e9c7a3eeb9c365a00910b02ef062065004cf1c3f594882992: Status 404 returned error can't find the container with id 7cc5b7b497dd6b2e9c7a3eeb9c365a00910b02ef062065004cf1c3f594882992
Jul 21 16:59:33.864017 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:33.863926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc" event={"ID":"f1a8365a-d37e-4e71-93cd-2b4d75c2544b","Type":"ContainerStarted","Data":"f44079f32b788d749eec4960d197f196cebe8357b69d82db4e293ff1d6d22fa1"}
Jul 21 16:59:33.864017 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:33.863980    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc" event={"ID":"f1a8365a-d37e-4e71-93cd-2b4d75c2544b","Type":"ContainerStarted","Data":"7cc5b7b497dd6b2e9c7a3eeb9c365a00910b02ef062065004cf1c3f594882992"}
Jul 21 16:59:33.881023 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:33.880952    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc" podStartSLOduration=1.880933453 podStartE2EDuration="1.880933453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:59:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:59:33.879405403 +0000 UTC m=+1869.378766647" watchObservedRunningTime="2026-07-21 16:59:33.880933453 +0000 UTC m=+1869.380294764"
Jul 21 16:59:34.870649 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:34.870609    2580 generic.go:356] "Generic (PLEG): container finished" podID="f1a8365a-d37e-4e71-93cd-2b4d75c2544b" containerID="f44079f32b788d749eec4960d197f196cebe8357b69d82db4e293ff1d6d22fa1" exitCode=0
Jul 21 16:59:34.871111 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:34.870701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc" event={"ID":"f1a8365a-d37e-4e71-93cd-2b4d75c2544b","Type":"ContainerDied","Data":"f44079f32b788d749eec4960d197f196cebe8357b69d82db4e293ff1d6d22fa1"}
Jul 21 16:59:36.003985 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:36.003959    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"
Jul 21 16:59:36.042716 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:36.042680    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1a8365a-d37e-4e71-93cd-2b4d75c2544b-maas-api-service-ca\") pod \"f1a8365a-d37e-4e71-93cd-2b4d75c2544b\" (UID: \"f1a8365a-d37e-4e71-93cd-2b4d75c2544b\") "
Jul 21 16:59:36.043065 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:36.043038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1a8365a-d37e-4e71-93cd-2b4d75c2544b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f1a8365a-d37e-4e71-93cd-2b4d75c2544b" (UID: "f1a8365a-d37e-4e71-93cd-2b4d75c2544b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:59:36.143564 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:36.143451    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1a8365a-d37e-4e71-93cd-2b4d75c2544b-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:59:36.880627 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:36.880592    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"
Jul 21 16:59:36.880811 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:36.880632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc" event={"ID":"f1a8365a-d37e-4e71-93cd-2b4d75c2544b","Type":"ContainerDied","Data":"7cc5b7b497dd6b2e9c7a3eeb9c365a00910b02ef062065004cf1c3f594882992"}
Jul 21 16:59:36.880811 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:36.880712    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cc5b7b497dd6b2e9c7a3eeb9c365a00910b02ef062065004cf1c3f594882992"
Jul 21 16:59:43.488643 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.488609    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"]
Jul 21 16:59:43.489136 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.488876    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl" podUID="baa8032b-613a-493c-8c6e-a75f2ef35b33" containerName="maas-api" containerID="cri-o://3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c" gracePeriod=30
Jul 21 16:59:43.735159 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.735127    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:43.813761 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.813671    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa8032b-613a-493c-8c6e-a75f2ef35b33-openshift-service-ca\") pod \"baa8032b-613a-493c-8c6e-a75f2ef35b33\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") "
Jul 21 16:59:43.813948 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.813798    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxww8\" (UniqueName: \"kubernetes.io/projected/baa8032b-613a-493c-8c6e-a75f2ef35b33-kube-api-access-lxww8\") pod \"baa8032b-613a-493c-8c6e-a75f2ef35b33\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") "
Jul 21 16:59:43.813948 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.813842    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa8032b-613a-493c-8c6e-a75f2ef35b33-maas-api-tls\") pod \"baa8032b-613a-493c-8c6e-a75f2ef35b33\" (UID: \"baa8032b-613a-493c-8c6e-a75f2ef35b33\") "
Jul 21 16:59:43.814121 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.814097    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baa8032b-613a-493c-8c6e-a75f2ef35b33-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "baa8032b-613a-493c-8c6e-a75f2ef35b33" (UID: "baa8032b-613a-493c-8c6e-a75f2ef35b33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 16:59:43.816089 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.816053    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa8032b-613a-493c-8c6e-a75f2ef35b33-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "baa8032b-613a-493c-8c6e-a75f2ef35b33" (UID: "baa8032b-613a-493c-8c6e-a75f2ef35b33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 16:59:43.816216 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.816192    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baa8032b-613a-493c-8c6e-a75f2ef35b33-kube-api-access-lxww8" (OuterVolumeSpecName: "kube-api-access-lxww8") pod "baa8032b-613a-493c-8c6e-a75f2ef35b33" (UID: "baa8032b-613a-493c-8c6e-a75f2ef35b33"). InnerVolumeSpecName "kube-api-access-lxww8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 16:59:43.913031 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.912995    2580 generic.go:356] "Generic (PLEG): container finished" podID="baa8032b-613a-493c-8c6e-a75f2ef35b33" containerID="3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c" exitCode=0
Jul 21 16:59:43.913212 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.913058    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"
Jul 21 16:59:43.913212 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.913078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl" event={"ID":"baa8032b-613a-493c-8c6e-a75f2ef35b33","Type":"ContainerDied","Data":"3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c"}
Jul 21 16:59:43.913212 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.913149    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl" event={"ID":"baa8032b-613a-493c-8c6e-a75f2ef35b33","Type":"ContainerDied","Data":"4e29d6216c014f201f23a410e5f3b4d6d3fa5d479c5fd4d9a0c4b0f74a9f5bbf"}
Jul 21 16:59:43.913212 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.913173    2580 scope.go:117] "RemoveContainer" containerID="3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c"
Jul 21 16:59:43.914445 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.914427    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxww8\" (UniqueName: \"kubernetes.io/projected/baa8032b-613a-493c-8c6e-a75f2ef35b33-kube-api-access-lxww8\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:59:43.914549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.914449    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa8032b-613a-493c-8c6e-a75f2ef35b33-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:59:43.914549 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.914460    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa8032b-613a-493c-8c6e-a75f2ef35b33-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 16:59:43.922725 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.922656    2580 scope.go:117] "RemoveContainer" containerID="3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c"
Jul 21 16:59:43.922994 ip-10-0-141-158 kubenswrapper[2580]: E0721 16:59:43.922975    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c\": container with ID starting with 3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c not found: ID does not exist" containerID="3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c"
Jul 21 16:59:43.923055 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.923006    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c"} err="failed to get container status \"3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c\": rpc error: code = NotFound desc = could not find container \"3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c\": container with ID starting with 3073b619315a7768c872d8c2e7838e1cbdcc038b43fc3fd606511bffb915a16c not found: ID does not exist"
Jul 21 16:59:43.934921 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.934892    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"]
Jul 21 16:59:43.938493 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:43.938463    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13ffe089-6df6994c64-4fjdl"]
Jul 21 16:59:44.939086 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:44.939054    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baa8032b-613a-493c-8c6e-a75f2ef35b33" path="/var/lib/kubelet/pods/baa8032b-613a-493c-8c6e-a75f2ef35b33/volumes"
Jul 21 16:59:48.012402 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:48.011256    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"]
Jul 21 16:59:48.016038 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:48.016006    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-13ffe089-5zwvc"]
Jul 21 16:59:48.939013 ip-10-0-141-158 kubenswrapper[2580]: I0721 16:59:48.938972    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1a8365a-d37e-4e71-93cd-2b4d75c2544b" path="/var/lib/kubelet/pods/f1a8365a-d37e-4e71-93cd-2b4d75c2544b/volumes"
Jul 21 17:00:00.137403 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.137363    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r"]
Jul 21 17:00:00.137940 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.137823    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa8032b-613a-493c-8c6e-a75f2ef35b33" containerName="maas-api"
Jul 21 17:00:00.137940 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.137841    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa8032b-613a-493c-8c6e-a75f2ef35b33" containerName="maas-api"
Jul 21 17:00:00.137940 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.137862    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1a8365a-d37e-4e71-93cd-2b4d75c2544b" containerName="revoke-keys"
Jul 21 17:00:00.137940 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.137870    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1a8365a-d37e-4e71-93cd-2b4d75c2544b" containerName="revoke-keys"
Jul 21 17:00:00.138149 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.137947    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="baa8032b-613a-493c-8c6e-a75f2ef35b33" containerName="maas-api"
Jul 21 17:00:00.138149 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.137958    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1a8365a-d37e-4e71-93cd-2b4d75c2544b" containerName="revoke-keys"
Jul 21 17:00:00.140889 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.140868    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r"
Jul 21 17:00:00.143325 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.143305    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 17:00:00.156593 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.156567    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r"]
Jul 21 17:00:00.158056 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.158037    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r"
Jul 21 17:00:00.289221 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.289108    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r"]
Jul 21 17:00:00.292180 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:00:00.292145    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55dc8451_4a20_4b99_9712_c88bac8ebfa3.slice/crio-c241407c4030ad6d3ad7530a10f0c58788756c3bcbe1d9ca2fbffd4c6d9c0461 WatchSource:0}: Error finding container c241407c4030ad6d3ad7530a10f0c58788756c3bcbe1d9ca2fbffd4c6d9c0461: Status 404 returned error can't find the container with id c241407c4030ad6d3ad7530a10f0c58788756c3bcbe1d9ca2fbffd4c6d9c0461
Jul 21 17:00:00.988327 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.988286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r" event={"ID":"55dc8451-4a20-4b99-9712-c88bac8ebfa3","Type":"ContainerStarted","Data":"1f1c70cd42e46b66d7c78e3a5a6e2ac81bc6158029be35e23c5cc5a782f1473e"}
Jul 21 17:00:00.988567 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:00.988341    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r" event={"ID":"55dc8451-4a20-4b99-9712-c88bac8ebfa3","Type":"ContainerStarted","Data":"c241407c4030ad6d3ad7530a10f0c58788756c3bcbe1d9ca2fbffd4c6d9c0461"}
Jul 21 17:00:01.035639 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:01.035573    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r" podStartSLOduration=1.035552144 podStartE2EDuration="1.035552144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:00:01.035419387 +0000 UTC m=+1896.534780634" watchObservedRunningTime="2026-07-21 17:00:01.035552144 +0000 UTC m=+1896.534913387"
Jul 21 17:00:01.994220 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:01.994185    2580 generic.go:356] "Generic (PLEG): container finished" podID="55dc8451-4a20-4b99-9712-c88bac8ebfa3" containerID="1f1c70cd42e46b66d7c78e3a5a6e2ac81bc6158029be35e23c5cc5a782f1473e" exitCode=0
Jul 21 17:00:01.994652 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:01.994281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r" event={"ID":"55dc8451-4a20-4b99-9712-c88bac8ebfa3","Type":"ContainerDied","Data":"1f1c70cd42e46b66d7c78e3a5a6e2ac81bc6158029be35e23c5cc5a782f1473e"}
Jul 21 17:00:03.130176 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:03.130153    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r"
Jul 21 17:00:04.004232 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:04.004203    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r"
Jul 21 17:00:04.004232 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:04.004231    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-kn54r" event={"ID":"55dc8451-4a20-4b99-9712-c88bac8ebfa3","Type":"ContainerDied","Data":"c241407c4030ad6d3ad7530a10f0c58788756c3bcbe1d9ca2fbffd4c6d9c0461"}
Jul 21 17:00:04.004451 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:04.004276    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c241407c4030ad6d3ad7530a10f0c58788756c3bcbe1d9ca2fbffd4c6d9c0461"
Jul 21 17:00:17.376855 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.376823    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"]
Jul 21 17:00:17.377477 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.377173    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55dc8451-4a20-4b99-9712-c88bac8ebfa3" containerName="cleanup"
Jul 21 17:00:17.377477 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.377190    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="55dc8451-4a20-4b99-9712-c88bac8ebfa3" containerName="cleanup"
Jul 21 17:00:17.377477 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.377265    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="55dc8451-4a20-4b99-9712-c88bac8ebfa3" containerName="cleanup"
Jul 21 17:00:17.380116 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.380090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:17.382646 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.382620    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:00:17.382805 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.382659    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:00:17.382805 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.382659    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-299dfdef-serving-cert\""
Jul 21 17:00:17.383724 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.383706    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9g9wv\""
Jul 21 17:00:17.389775 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.389753    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"]
Jul 21 17:00:17.507845 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.507805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-openshift-service-ca\") pod \"maas-api-e2e-mt-299dfdef-58954d4855-tf574\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:17.507845 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.507846    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzk9b\" (UniqueName: \"kubernetes.io/projected/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-kube-api-access-vzk9b\") pod \"maas-api-e2e-mt-299dfdef-58954d4855-tf574\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:17.508071 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.507945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-maas-api-tls\") pod \"maas-api-e2e-mt-299dfdef-58954d4855-tf574\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:17.608881 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.608843    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-openshift-service-ca\") pod \"maas-api-e2e-mt-299dfdef-58954d4855-tf574\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:17.608881 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.608882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzk9b\" (UniqueName: \"kubernetes.io/projected/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-kube-api-access-vzk9b\") pod \"maas-api-e2e-mt-299dfdef-58954d4855-tf574\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:17.609152 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.608928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-maas-api-tls\") pod \"maas-api-e2e-mt-299dfdef-58954d4855-tf574\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:17.609152 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:00:17.609043    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-serving-cert: secret "maas-api-e2e-mt-299dfdef-serving-cert" not found
Jul 21 17:00:17.609152 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:00:17.609124    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-maas-api-tls podName:f0bc93a9-41a9-4254-8d81-18f45cb5f7b2 nodeName:}" failed. No retries permitted until 2026-07-21 17:00:18.109107535 +0000 UTC m=+1913.608468759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-maas-api-tls") pod "maas-api-e2e-mt-299dfdef-58954d4855-tf574" (UID: "f0bc93a9-41a9-4254-8d81-18f45cb5f7b2") : secret "maas-api-e2e-mt-299dfdef-serving-cert" not found
Jul 21 17:00:17.609568 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.609548    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-openshift-service-ca\") pod \"maas-api-e2e-mt-299dfdef-58954d4855-tf574\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:17.618331 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:17.618298    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzk9b\" (UniqueName: \"kubernetes.io/projected/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-kube-api-access-vzk9b\") pod \"maas-api-e2e-mt-299dfdef-58954d4855-tf574\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:18.113893 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:18.113859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-maas-api-tls\") pod \"maas-api-e2e-mt-299dfdef-58954d4855-tf574\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:18.116343 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:18.116320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-maas-api-tls\") pod \"maas-api-e2e-mt-299dfdef-58954d4855-tf574\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:18.292602 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:18.292552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:18.436089 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:18.436063    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"]
Jul 21 17:00:18.438708 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:00:18.438681    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0bc93a9_41a9_4254_8d81_18f45cb5f7b2.slice/crio-4674e715870fbdaa0122ea978103c6002cb688ac37d95f341421544750bc17ef WatchSource:0}: Error finding container 4674e715870fbdaa0122ea978103c6002cb688ac37d95f341421544750bc17ef: Status 404 returned error can't find the container with id 4674e715870fbdaa0122ea978103c6002cb688ac37d95f341421544750bc17ef
Jul 21 17:00:19.076188 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:19.076081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574" event={"ID":"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2","Type":"ContainerStarted","Data":"4674e715870fbdaa0122ea978103c6002cb688ac37d95f341421544750bc17ef"}
Jul 21 17:00:20.082302 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:20.082251    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574" event={"ID":"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2","Type":"ContainerStarted","Data":"1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e"}
Jul 21 17:00:20.082734 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:20.082398    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:20.100051 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:20.099995    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574" podStartSLOduration=2.11094333 podStartE2EDuration="3.099975887s" totalImagesPullingTime="989.032557ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:00:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:00:20.099341118 +0000 UTC m=+1915.598702362" watchObservedRunningTime="2026-07-21 17:00:20.099975887 +0000 UTC m=+1915.599337132"
Jul 21 17:00:26.092538 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:26.092492    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:00:42.674855 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.674822    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"]
Jul 21 17:00:42.678150 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.678131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.680802 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.680776    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-299dfdef\"/\"kube-root-ca.crt\""
Jul 21 17:00:42.681803 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.681780    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-299dfdef\"/\"e2e-lifecycle-model-299dfdef-kserve-self-signed-certs\""
Jul 21 17:00:42.681803 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.681793    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-299dfdef\"/\"default-dockercfg-qdljj\""
Jul 21 17:00:42.681968 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.681823    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-299dfdef\"/\"openshift-service-ca.crt\""
Jul 21 17:00:42.690554 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.689530    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"]
Jul 21 17:00:42.828509 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.828473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-home\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.828712 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.828535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-model-cache\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.828712 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.828570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhx9t\" (UniqueName: \"kubernetes.io/projected/914267db-25fe-4904-9eac-4b663da8afc0-kube-api-access-fhx9t\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.828712 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.828631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-dshm\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.828712 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.828661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/914267db-25fe-4904-9eac-4b663da8afc0-tls-certs\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.828873 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.828726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-tmp-dir\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.930156 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.930061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-tmp-dir\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.930156 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.930117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-home\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.930156 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.930133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-model-cache\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.930441 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.930269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhx9t\" (UniqueName: \"kubernetes.io/projected/914267db-25fe-4904-9eac-4b663da8afc0-kube-api-access-fhx9t\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.930441 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.930315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-dshm\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.930441 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.930334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/914267db-25fe-4904-9eac-4b663da8afc0-tls-certs\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.930633 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.930498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-model-cache\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.930633 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.930568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-home\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.930633 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.930615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-tmp-dir\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.932810 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.932792    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-dshm\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.933009 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.932993    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/914267db-25fe-4904-9eac-4b663da8afc0-tls-certs\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.940141 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.940114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhx9t\" (UniqueName: \"kubernetes.io/projected/914267db-25fe-4904-9eac-4b663da8afc0-kube-api-access-fhx9t\") pod \"e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") " pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:42.987863 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:42.987833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:43.129377 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:43.129351    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"]
Jul 21 17:00:43.131905 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:00:43.131881    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod914267db_25fe_4904_9eac_4b663da8afc0.slice/crio-47eebcc3ad297fba9d797dc959a82cad463dc8f1d535eb79de94ad4f42f150dd WatchSource:0}: Error finding container 47eebcc3ad297fba9d797dc959a82cad463dc8f1d535eb79de94ad4f42f150dd: Status 404 returned error can't find the container with id 47eebcc3ad297fba9d797dc959a82cad463dc8f1d535eb79de94ad4f42f150dd
Jul 21 17:00:43.189341 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:43.189308    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd" event={"ID":"914267db-25fe-4904-9eac-4b663da8afc0","Type":"ContainerStarted","Data":"47eebcc3ad297fba9d797dc959a82cad463dc8f1d535eb79de94ad4f42f150dd"}
Jul 21 17:00:44.196422 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:44.196390    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd" event={"ID":"914267db-25fe-4904-9eac-4b663da8afc0","Type":"ContainerStarted","Data":"02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869"}
Jul 21 17:00:44.217032 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:44.216988    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd" podStartSLOduration=2.216970917 podStartE2EDuration="2.216970917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:00:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:00:44.21580681 +0000 UTC m=+1939.715168070" watchObservedRunningTime="2026-07-21 17:00:44.216970917 +0000 UTC m=+1939.716332161"
Jul 21 17:00:52.988318 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:52.988278    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:52.988318 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:52.988314    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:53.000761 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:53.000736    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:00:53.250160 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:00:53.250086    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:01:05.461549 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:05.461494    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"]
Jul 21 17:01:05.464931 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:05.464910    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"
Jul 21 17:01:05.467611 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:05.467591    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 17:01:05.473025 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:05.473000    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"]
Jul 21 17:01:05.524656 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:05.524629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-299dfdef-d29sj\" (UID: \"9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"
Jul 21 17:01:05.625982 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:05.625932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-299dfdef-d29sj\" (UID: \"9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"
Jul 21 17:01:05.626607 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:05.626588    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-299dfdef-d29sj\" (UID: \"9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"
Jul 21 17:01:05.776036 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:05.775951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"
Jul 21 17:01:05.919825 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:01:05.919797    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1ed5e5_8c8b_44d2_9af3_7512642ce8a6.slice/crio-8f4bb72c195ec62d5ad8039d0613450973e76833704234ac3269bcbd1fcb9d60 WatchSource:0}: Error finding container 8f4bb72c195ec62d5ad8039d0613450973e76833704234ac3269bcbd1fcb9d60: Status 404 returned error can't find the container with id 8f4bb72c195ec62d5ad8039d0613450973e76833704234ac3269bcbd1fcb9d60
Jul 21 17:01:05.923212 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:05.923189    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"]
Jul 21 17:01:06.298981 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:06.298947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj" event={"ID":"9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6","Type":"ContainerStarted","Data":"7a1642e74019a2d6dcd971bd7464ed360ac5578b1aef1aedbc7efdb03d43beff"}
Jul 21 17:01:06.299148 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:06.299002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj" event={"ID":"9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6","Type":"ContainerStarted","Data":"8f4bb72c195ec62d5ad8039d0613450973e76833704234ac3269bcbd1fcb9d60"}
Jul 21 17:01:06.330484 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:06.330440    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj" podStartSLOduration=1.330424286 podStartE2EDuration="1.330424286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:01:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:01:06.326634921 +0000 UTC m=+1961.825996165" watchObservedRunningTime="2026-07-21 17:01:06.330424286 +0000 UTC m=+1961.829785530"
Jul 21 17:01:07.304563 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:07.304506    2580 generic.go:356] "Generic (PLEG): container finished" podID="9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6" containerID="7a1642e74019a2d6dcd971bd7464ed360ac5578b1aef1aedbc7efdb03d43beff" exitCode=0
Jul 21 17:01:07.305004 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:07.304604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj" event={"ID":"9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6","Type":"ContainerDied","Data":"7a1642e74019a2d6dcd971bd7464ed360ac5578b1aef1aedbc7efdb03d43beff"}
Jul 21 17:01:08.438565 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:08.438539    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"
Jul 21 17:01:08.548681 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:08.548651    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6-maas-api-service-ca\") pod \"9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6\" (UID: \"9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6\") "
Jul 21 17:01:08.548978 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:08.548958    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6" (UID: "9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:01:08.650123 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:08.650049    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:01:09.316215 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:09.316138    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"
Jul 21 17:01:09.316362 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:09.316138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj" event={"ID":"9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6","Type":"ContainerDied","Data":"8f4bb72c195ec62d5ad8039d0613450973e76833704234ac3269bcbd1fcb9d60"}
Jul 21 17:01:09.316362 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:09.316279    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f4bb72c195ec62d5ad8039d0613450973e76833704234ac3269bcbd1fcb9d60"
Jul 21 17:01:21.120323 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.120251    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"]
Jul 21 17:01:21.120755 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.120494    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574" podUID="f0bc93a9-41a9-4254-8d81-18f45cb5f7b2" containerName="maas-api" containerID="cri-o://1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e" gracePeriod=30
Jul 21 17:01:21.368963 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.368934    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:01:21.370735 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.370671    2580 generic.go:356] "Generic (PLEG): container finished" podID="f0bc93a9-41a9-4254-8d81-18f45cb5f7b2" containerID="1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e" exitCode=0
Jul 21 17:01:21.370735 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.370706    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"
Jul 21 17:01:21.370885 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.370751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574" event={"ID":"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2","Type":"ContainerDied","Data":"1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e"}
Jul 21 17:01:21.370885 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.370819    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574" event={"ID":"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2","Type":"ContainerDied","Data":"4674e715870fbdaa0122ea978103c6002cb688ac37d95f341421544750bc17ef"}
Jul 21 17:01:21.370885 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.370843    2580 scope.go:117] "RemoveContainer" containerID="1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e"
Jul 21 17:01:21.380126 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.380107    2580 scope.go:117] "RemoveContainer" containerID="1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e"
Jul 21 17:01:21.380415 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:01:21.380397    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e\": container with ID starting with 1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e not found: ID does not exist" containerID="1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e"
Jul 21 17:01:21.380481 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.380426    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e"} err="failed to get container status \"1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e\": rpc error: code = NotFound desc = could not find container \"1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e\": container with ID starting with 1e24475eb245e287d792abb20dc986d9b6cb5ff6d0b33106885017220d5e1a6e not found: ID does not exist"
Jul 21 17:01:21.471588 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.471552    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzk9b\" (UniqueName: \"kubernetes.io/projected/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-kube-api-access-vzk9b\") pod \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") "
Jul 21 17:01:21.471760 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.471617    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-maas-api-tls\") pod \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") "
Jul 21 17:01:21.471760 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.471703    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-openshift-service-ca\") pod \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\" (UID: \"f0bc93a9-41a9-4254-8d81-18f45cb5f7b2\") "
Jul 21 17:01:21.472060 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.472031    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f0bc93a9-41a9-4254-8d81-18f45cb5f7b2" (UID: "f0bc93a9-41a9-4254-8d81-18f45cb5f7b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:01:21.473928 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.473908    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f0bc93a9-41a9-4254-8d81-18f45cb5f7b2" (UID: "f0bc93a9-41a9-4254-8d81-18f45cb5f7b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:01:21.473928 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.473919    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-kube-api-access-vzk9b" (OuterVolumeSpecName: "kube-api-access-vzk9b") pod "f0bc93a9-41a9-4254-8d81-18f45cb5f7b2" (UID: "f0bc93a9-41a9-4254-8d81-18f45cb5f7b2"). InnerVolumeSpecName "kube-api-access-vzk9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:01:21.573098 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.573064    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:01:21.573098 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.573094    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzk9b\" (UniqueName: \"kubernetes.io/projected/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-kube-api-access-vzk9b\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:01:21.573098 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.573103    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:01:21.704138 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.704102    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"]
Jul 21 17:01:21.728724 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:21.728671    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-299dfdef-58954d4855-tf574"]
Jul 21 17:01:22.940006 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:22.939972    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0bc93a9-41a9-4254-8d81-18f45cb5f7b2" path="/var/lib/kubelet/pods/f0bc93a9-41a9-4254-8d81-18f45cb5f7b2/volumes"
Jul 21 17:01:25.036624 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:25.036594    2580 scope.go:117] "RemoveContainer" containerID="92d784042b138f6432a8cae49836825d2bbcb083adf76e987b6b080663756645"
Jul 21 17:01:25.658732 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:25.658692    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"]
Jul 21 17:01:25.672250 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:25.672218    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-299dfdef-d29sj"]
Jul 21 17:01:26.938926 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:26.938894    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6" path="/var/lib/kubelet/pods/9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6/volumes"
Jul 21 17:01:32.866873 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:32.866839    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"]
Jul 21 17:01:47.888852 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:47.888775    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd" podUID="914267db-25fe-4904-9eac-4b663da8afc0" containerName="main" containerID="cri-o://02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869" gracePeriod=45
Jul 21 17:01:48.143883 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.143822    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:01:48.202223 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202186    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-model-cache\") pod \"914267db-25fe-4904-9eac-4b663da8afc0\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") "
Jul 21 17:01:48.202408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202236    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/914267db-25fe-4904-9eac-4b663da8afc0-tls-certs\") pod \"914267db-25fe-4904-9eac-4b663da8afc0\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") "
Jul 21 17:01:48.202408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202279    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhx9t\" (UniqueName: \"kubernetes.io/projected/914267db-25fe-4904-9eac-4b663da8afc0-kube-api-access-fhx9t\") pod \"914267db-25fe-4904-9eac-4b663da8afc0\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") "
Jul 21 17:01:48.202408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202354    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-tmp-dir\") pod \"914267db-25fe-4904-9eac-4b663da8afc0\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") "
Jul 21 17:01:48.202408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202405    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-dshm\") pod \"914267db-25fe-4904-9eac-4b663da8afc0\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") "
Jul 21 17:01:48.202664 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202435    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-home\") pod \"914267db-25fe-4904-9eac-4b663da8afc0\" (UID: \"914267db-25fe-4904-9eac-4b663da8afc0\") "
Jul 21 17:01:48.202664 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202499    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-model-cache" (OuterVolumeSpecName: "model-cache") pod "914267db-25fe-4904-9eac-4b663da8afc0" (UID: "914267db-25fe-4904-9eac-4b663da8afc0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:01:48.202664 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202607    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "914267db-25fe-4904-9eac-4b663da8afc0" (UID: "914267db-25fe-4904-9eac-4b663da8afc0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:01:48.202833 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202774    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-home" (OuterVolumeSpecName: "home") pod "914267db-25fe-4904-9eac-4b663da8afc0" (UID: "914267db-25fe-4904-9eac-4b663da8afc0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:01:48.202833 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202790    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-model-cache\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:01:48.202833 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.202805    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-tmp-dir\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:01:48.204691 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.204669    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914267db-25fe-4904-9eac-4b663da8afc0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "914267db-25fe-4904-9eac-4b663da8afc0" (UID: "914267db-25fe-4904-9eac-4b663da8afc0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:01:48.204808 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.204671    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-dshm" (OuterVolumeSpecName: "dshm") pod "914267db-25fe-4904-9eac-4b663da8afc0" (UID: "914267db-25fe-4904-9eac-4b663da8afc0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:01:48.204879 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.204862    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914267db-25fe-4904-9eac-4b663da8afc0-kube-api-access-fhx9t" (OuterVolumeSpecName: "kube-api-access-fhx9t") pod "914267db-25fe-4904-9eac-4b663da8afc0" (UID: "914267db-25fe-4904-9eac-4b663da8afc0"). InnerVolumeSpecName "kube-api-access-fhx9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:01:48.303886 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.303846    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/914267db-25fe-4904-9eac-4b663da8afc0-tls-certs\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:01:48.303886 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.303879    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhx9t\" (UniqueName: \"kubernetes.io/projected/914267db-25fe-4904-9eac-4b663da8afc0-kube-api-access-fhx9t\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:01:48.303886 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.303889    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-dshm\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:01:48.303886 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.303898    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/914267db-25fe-4904-9eac-4b663da8afc0-home\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:01:48.505266 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.505229    2580 generic.go:356] "Generic (PLEG): container finished" podID="914267db-25fe-4904-9eac-4b663da8afc0" containerID="02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869" exitCode=0
Jul 21 17:01:48.505266 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.505266    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd" event={"ID":"914267db-25fe-4904-9eac-4b663da8afc0","Type":"ContainerDied","Data":"02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869"}
Jul 21 17:01:48.505571 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.505278    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"
Jul 21 17:01:48.505571 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.505292    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd" event={"ID":"914267db-25fe-4904-9eac-4b663da8afc0","Type":"ContainerDied","Data":"47eebcc3ad297fba9d797dc959a82cad463dc8f1d535eb79de94ad4f42f150dd"}
Jul 21 17:01:48.505571 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.505308    2580 scope.go:117] "RemoveContainer" containerID="02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869"
Jul 21 17:01:48.515288 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.515269    2580 scope.go:117] "RemoveContainer" containerID="02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869"
Jul 21 17:01:48.515650 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:01:48.515627    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869\": container with ID starting with 02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869 not found: ID does not exist" containerID="02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869"
Jul 21 17:01:48.515734 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.515659    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869"} err="failed to get container status \"02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869\": rpc error: code = NotFound desc = could not find container \"02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869\": container with ID starting with 02b2fafadf06cde790e1972b742b72277c078852b8c8390e133d9fd011ed7869 not found: ID does not exist"
Jul 21 17:01:48.547411 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.547373    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"]
Jul 21 17:01:48.553991 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.553961    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-299dfdef/e2e-lifecycle-model-299dfdef-kserve-6c6cf6d69f-dmrrd"]
Jul 21 17:01:48.939194 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:01:48.939160    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="914267db-25fe-4904-9eac-4b663da8afc0" path="/var/lib/kubelet/pods/914267db-25fe-4904-9eac-4b663da8afc0/volumes"
Jul 21 17:02:25.060810 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:02:25.060776    2580 scope.go:117] "RemoveContainer" containerID="619592a85c027f4ee0675b514f2fc03cbac634b12e57cd97fb29a11b6f80a16d"
Jul 21 17:03:25.082599 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:25.082572    2580 scope.go:117] "RemoveContainer" containerID="375d5dd5b5c04619444a5e0ff48a4ef4fd09541e7bdf20dc8e737708368c71bf"
Jul 21 17:03:25.090239 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:25.090210    2580 scope.go:117] "RemoveContainer" containerID="65a8cb11e66c710683a42f01f332145ffb92c0aea285028cd6f662dc81b5c330"
Jul 21 17:03:27.254936 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.254906    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"]
Jul 21 17:03:27.255305 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.255230    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914267db-25fe-4904-9eac-4b663da8afc0" containerName="main"
Jul 21 17:03:27.255305 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.255241    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="914267db-25fe-4904-9eac-4b663da8afc0" containerName="main"
Jul 21 17:03:27.255305 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.255255    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0bc93a9-41a9-4254-8d81-18f45cb5f7b2" containerName="maas-api"
Jul 21 17:03:27.255305 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.255262    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0bc93a9-41a9-4254-8d81-18f45cb5f7b2" containerName="maas-api"
Jul 21 17:03:27.255305 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.255273    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6" containerName="revoke-keys"
Jul 21 17:03:27.255305 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.255279    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6" containerName="revoke-keys"
Jul 21 17:03:27.255506 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.255330    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="914267db-25fe-4904-9eac-4b663da8afc0" containerName="main"
Jul 21 17:03:27.255506 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.255340    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0bc93a9-41a9-4254-8d81-18f45cb5f7b2" containerName="maas-api"
Jul 21 17:03:27.255506 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.255345    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e1ed5e5-8c8b-44d2-9af3-7512642ce8a6" containerName="revoke-keys"
Jul 21 17:03:27.259976 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.259958    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:27.268263 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.268242    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-981b1e-serving-cert\""
Jul 21 17:03:27.268503 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.268487    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:03:27.268642 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.268616    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9g9wv\""
Jul 21 17:03:27.268781 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.268762    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:03:27.275315 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.275293    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"]
Jul 21 17:03:27.365037 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.365006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c5938d-e9c2-4108-91ad-773de8d04bb8-maas-api-tls\") pod \"maas-api-e2e-inf-a-981b1e-5d47766b86-2j747\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:27.365207 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.365059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd9l6\" (UniqueName: \"kubernetes.io/projected/02c5938d-e9c2-4108-91ad-773de8d04bb8-kube-api-access-cd9l6\") pod \"maas-api-e2e-inf-a-981b1e-5d47766b86-2j747\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:27.365207 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.365180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c5938d-e9c2-4108-91ad-773de8d04bb8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-981b1e-5d47766b86-2j747\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:27.466374 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.466332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c5938d-e9c2-4108-91ad-773de8d04bb8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-981b1e-5d47766b86-2j747\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:27.466610 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.466382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c5938d-e9c2-4108-91ad-773de8d04bb8-maas-api-tls\") pod \"maas-api-e2e-inf-a-981b1e-5d47766b86-2j747\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:27.466610 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.466422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd9l6\" (UniqueName: \"kubernetes.io/projected/02c5938d-e9c2-4108-91ad-773de8d04bb8-kube-api-access-cd9l6\") pod \"maas-api-e2e-inf-a-981b1e-5d47766b86-2j747\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:27.466610 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:03:27.466587    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-serving-cert: secret "maas-api-e2e-inf-a-981b1e-serving-cert" not found
Jul 21 17:03:27.466788 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:03:27.466650    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c5938d-e9c2-4108-91ad-773de8d04bb8-maas-api-tls podName:02c5938d-e9c2-4108-91ad-773de8d04bb8 nodeName:}" failed. No retries permitted until 2026-07-21 17:03:27.966635777 +0000 UTC m=+2103.465997001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/02c5938d-e9c2-4108-91ad-773de8d04bb8-maas-api-tls") pod "maas-api-e2e-inf-a-981b1e-5d47766b86-2j747" (UID: "02c5938d-e9c2-4108-91ad-773de8d04bb8") : secret "maas-api-e2e-inf-a-981b1e-serving-cert" not found
Jul 21 17:03:27.466971 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.466950    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c5938d-e9c2-4108-91ad-773de8d04bb8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-981b1e-5d47766b86-2j747\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:27.479374 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.479350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd9l6\" (UniqueName: \"kubernetes.io/projected/02c5938d-e9c2-4108-91ad-773de8d04bb8-kube-api-access-cd9l6\") pod \"maas-api-e2e-inf-a-981b1e-5d47766b86-2j747\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:27.970825 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.970783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c5938d-e9c2-4108-91ad-773de8d04bb8-maas-api-tls\") pod \"maas-api-e2e-inf-a-981b1e-5d47766b86-2j747\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:27.973462 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:27.973436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c5938d-e9c2-4108-91ad-773de8d04bb8-maas-api-tls\") pod \"maas-api-e2e-inf-a-981b1e-5d47766b86-2j747\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:28.170782 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:28.170740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:28.304982 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:28.304832    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"]
Jul 21 17:03:28.307639 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:03:28.307610    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c5938d_e9c2_4108_91ad_773de8d04bb8.slice/crio-519da430be336691c151882470acf2dc274462ef196c2709f71e506935942795 WatchSource:0}: Error finding container 519da430be336691c151882470acf2dc274462ef196c2709f71e506935942795: Status 404 returned error can't find the container with id 519da430be336691c151882470acf2dc274462ef196c2709f71e506935942795
Jul 21 17:03:28.309023 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:28.309007    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 17:03:28.963601 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:28.963492    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747" event={"ID":"02c5938d-e9c2-4108-91ad-773de8d04bb8","Type":"ContainerStarted","Data":"a857ffcdbfd55b5a4dfb7dba00738af9da3b0ac427cecfb508a80a144810f173"}
Jul 21 17:03:28.963601 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:28.963607    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:28.963814 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:28.963627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747" event={"ID":"02c5938d-e9c2-4108-91ad-773de8d04bb8","Type":"ContainerStarted","Data":"519da430be336691c151882470acf2dc274462ef196c2709f71e506935942795"}
Jul 21 17:03:28.985066 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:28.985021    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747" podStartSLOduration=1.524343069 podStartE2EDuration="1.985005886s" totalImagesPullingTime="460.662817ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:28.983866307 +0000 UTC m=+2104.483227551" watchObservedRunningTime="2026-07-21 17:03:28.985005886 +0000 UTC m=+2104.484367130"
Jul 21 17:03:34.973704 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:34.973675    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:03:35.370199 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.370115    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"]
Jul 21 17:03:35.374143 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.374105    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.376757 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.376731    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-fe1af7-serving-cert\""
Jul 21 17:03:35.386025 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.385991    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"]
Jul 21 17:03:35.439905 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.439871    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7f34a70-d0c2-4501-a85e-056836f8e2b0-maas-api-tls\") pod \"maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.440102 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.439926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x2bv\" (UniqueName: \"kubernetes.io/projected/d7f34a70-d0c2-4501-a85e-056836f8e2b0-kube-api-access-7x2bv\") pod \"maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.440102 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.439954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7f34a70-d0c2-4501-a85e-056836f8e2b0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.540668 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.540629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7f34a70-d0c2-4501-a85e-056836f8e2b0-maas-api-tls\") pod \"maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.540854 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.540697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x2bv\" (UniqueName: \"kubernetes.io/projected/d7f34a70-d0c2-4501-a85e-056836f8e2b0-kube-api-access-7x2bv\") pod \"maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.540854 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.540721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7f34a70-d0c2-4501-a85e-056836f8e2b0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.541358 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.541340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7f34a70-d0c2-4501-a85e-056836f8e2b0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.543292 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.543274    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7f34a70-d0c2-4501-a85e-056836f8e2b0-maas-api-tls\") pod \"maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.550587 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.550561    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x2bv\" (UniqueName: \"kubernetes.io/projected/d7f34a70-d0c2-4501-a85e-056836f8e2b0-kube-api-access-7x2bv\") pod \"maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.686865 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.686825    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:35.825898 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.825873    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"]
Jul 21 17:03:35.828299 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:03:35.828255    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f34a70_d0c2_4501_a85e_056836f8e2b0.slice/crio-e817798f2a4cd6ea50e6ea5eda855fa840df5c667147a69aec166a720e13c403 WatchSource:0}: Error finding container e817798f2a4cd6ea50e6ea5eda855fa840df5c667147a69aec166a720e13c403: Status 404 returned error can't find the container with id e817798f2a4cd6ea50e6ea5eda855fa840df5c667147a69aec166a720e13c403
Jul 21 17:03:35.996381 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:35.996255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9" event={"ID":"d7f34a70-d0c2-4501-a85e-056836f8e2b0","Type":"ContainerStarted","Data":"e817798f2a4cd6ea50e6ea5eda855fa840df5c667147a69aec166a720e13c403"}
Jul 21 17:03:37.002184 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:37.002151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9" event={"ID":"d7f34a70-d0c2-4501-a85e-056836f8e2b0","Type":"ContainerStarted","Data":"03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea"}
Jul 21 17:03:37.002608 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:37.002282    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:37.040994 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:37.040930    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9" podStartSLOduration=1.476304525 podStartE2EDuration="2.040912344s" totalImagesPullingTime="564.607819ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:37.038352089 +0000 UTC m=+2112.537713332" watchObservedRunningTime="2026-07-21 17:03:37.040912344 +0000 UTC m=+2112.540273587"
Jul 21 17:03:43.012718 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:43.012686    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:03:50.070466 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.070432    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"]
Jul 21 17:03:50.073842 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.073819    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.077975 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.077946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-981b1e\"/\"kube-root-ca.crt\""
Jul 21 17:03:50.078089 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.078028    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-981b1e\"/\"openshift-service-ca.crt\""
Jul 21 17:03:50.078947 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.078928    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-981b1e\"/\"test-model-981b1e-kserve-self-signed-certs\""
Jul 21 17:03:50.080539 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.080500    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-981b1e\"/\"default-dockercfg-btdh7\""
Jul 21 17:03:50.086582 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.086556    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"]
Jul 21 17:03:50.162807 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.162765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-dshm\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.162807 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.162809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tmp-dir\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.163019 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.162915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tls-certs\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.163019 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.162952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-model-cache\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.163019 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.162982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-home\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.163116 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.163022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l67p6\" (UniqueName: \"kubernetes.io/projected/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-kube-api-access-l67p6\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.264312 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.264262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-home\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.264312 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.264323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l67p6\" (UniqueName: \"kubernetes.io/projected/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-kube-api-access-l67p6\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.264595 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.264359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-dshm\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.264595 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.264380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tmp-dir\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.264595 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.264450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tls-certs\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.264595 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.264484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-model-cache\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.264863 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.264701    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-home\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.264863 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.264795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tmp-dir\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.264863 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.264851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-model-cache\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.266905 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.266885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-dshm\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.267134 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.267118    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tls-certs\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.273201 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.273170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l67p6\" (UniqueName: \"kubernetes.io/projected/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-kube-api-access-l67p6\") pod \"test-model-981b1e-kserve-79776b6dd9-vvjxr\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") " pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.384973 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.384890    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:03:50.523564 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:50.523541    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"]
Jul 21 17:03:50.525419 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:03:50.525389    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8835e00_72bc_4262_a1fa_a32dfb69e0f5.slice/crio-c401699e7a1cea7f5827c02f576f1729b5d53ca7eeba7ac7df7aac842eccfa8f WatchSource:0}: Error finding container c401699e7a1cea7f5827c02f576f1729b5d53ca7eeba7ac7df7aac842eccfa8f: Status 404 returned error can't find the container with id c401699e7a1cea7f5827c02f576f1729b5d53ca7eeba7ac7df7aac842eccfa8f
Jul 21 17:03:51.067240 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:51.067198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr" event={"ID":"e8835e00-72bc-4262-a1fa-a32dfb69e0f5","Type":"ContainerStarted","Data":"1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994"}
Jul 21 17:03:51.067416 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:51.067249    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr" event={"ID":"e8835e00-72bc-4262-a1fa-a32dfb69e0f5","Type":"ContainerStarted","Data":"c401699e7a1cea7f5827c02f576f1729b5d53ca7eeba7ac7df7aac842eccfa8f"}
Jul 21 17:03:51.093754 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:03:51.093699    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr" podStartSLOduration=1.093682481 podStartE2EDuration="1.093682481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:03:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:51.089189356 +0000 UTC m=+2126.588550599" watchObservedRunningTime="2026-07-21 17:03:51.093682481 +0000 UTC m=+2126.593043730"
Jul 21 17:04:00.385357 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:04:00.385327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:04:00.386004 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:04:00.385624    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:04:00.397759 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:04:00.397737    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:04:01.123462 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:04:01.123429    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:05:14.591486 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:14.591454    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"]
Jul 21 17:05:26.080763 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.080729    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"]
Jul 21 17:05:26.082992 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.081031    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747" podUID="02c5938d-e9c2-4108-91ad-773de8d04bb8" containerName="maas-api" containerID="cri-o://a857ffcdbfd55b5a4dfb7dba00738af9da3b0ac427cecfb508a80a144810f173" gracePeriod=30
Jul 21 17:05:26.499380 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.499351    2580 generic.go:356] "Generic (PLEG): container finished" podID="02c5938d-e9c2-4108-91ad-773de8d04bb8" containerID="a857ffcdbfd55b5a4dfb7dba00738af9da3b0ac427cecfb508a80a144810f173" exitCode=0
Jul 21 17:05:26.499543 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.499422    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747" event={"ID":"02c5938d-e9c2-4108-91ad-773de8d04bb8","Type":"ContainerDied","Data":"a857ffcdbfd55b5a4dfb7dba00738af9da3b0ac427cecfb508a80a144810f173"}
Jul 21 17:05:26.519482 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.519466    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:05:26.583001 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.582981    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd9l6\" (UniqueName: \"kubernetes.io/projected/02c5938d-e9c2-4108-91ad-773de8d04bb8-kube-api-access-cd9l6\") pod \"02c5938d-e9c2-4108-91ad-773de8d04bb8\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") "
Jul 21 17:05:26.583121 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.583036    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c5938d-e9c2-4108-91ad-773de8d04bb8-openshift-service-ca\") pod \"02c5938d-e9c2-4108-91ad-773de8d04bb8\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") "
Jul 21 17:05:26.583187 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.583122    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c5938d-e9c2-4108-91ad-773de8d04bb8-maas-api-tls\") pod \"02c5938d-e9c2-4108-91ad-773de8d04bb8\" (UID: \"02c5938d-e9c2-4108-91ad-773de8d04bb8\") "
Jul 21 17:05:26.583435 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.583403    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02c5938d-e9c2-4108-91ad-773de8d04bb8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "02c5938d-e9c2-4108-91ad-773de8d04bb8" (UID: "02c5938d-e9c2-4108-91ad-773de8d04bb8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:05:26.585217 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.585192    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02c5938d-e9c2-4108-91ad-773de8d04bb8-kube-api-access-cd9l6" (OuterVolumeSpecName: "kube-api-access-cd9l6") pod "02c5938d-e9c2-4108-91ad-773de8d04bb8" (UID: "02c5938d-e9c2-4108-91ad-773de8d04bb8"). InnerVolumeSpecName "kube-api-access-cd9l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:05:26.585305 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.585287    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02c5938d-e9c2-4108-91ad-773de8d04bb8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "02c5938d-e9c2-4108-91ad-773de8d04bb8" (UID: "02c5938d-e9c2-4108-91ad-773de8d04bb8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:05:26.684657 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.684629    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd9l6\" (UniqueName: \"kubernetes.io/projected/02c5938d-e9c2-4108-91ad-773de8d04bb8-kube-api-access-cd9l6\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:26.684657 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.684654    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c5938d-e9c2-4108-91ad-773de8d04bb8-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:26.684810 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:26.684664    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c5938d-e9c2-4108-91ad-773de8d04bb8-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:27.504400 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:27.504370    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"
Jul 21 17:05:27.504890 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:27.504400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747" event={"ID":"02c5938d-e9c2-4108-91ad-773de8d04bb8","Type":"ContainerDied","Data":"519da430be336691c151882470acf2dc274462ef196c2709f71e506935942795"}
Jul 21 17:05:27.504890 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:27.504469    2580 scope.go:117] "RemoveContainer" containerID="a857ffcdbfd55b5a4dfb7dba00738af9da3b0ac427cecfb508a80a144810f173"
Jul 21 17:05:27.524834 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:27.524813    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"]
Jul 21 17:05:27.529347 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:27.529328    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-981b1e-5d47766b86-2j747"]
Jul 21 17:05:28.939418 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:28.939390    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02c5938d-e9c2-4108-91ad-773de8d04bb8" path="/var/lib/kubelet/pods/02c5938d-e9c2-4108-91ad-773de8d04bb8/volumes"
Jul 21 17:05:29.611917 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:29.611874    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr" podUID="e8835e00-72bc-4262-a1fa-a32dfb69e0f5" containerName="main" containerID="cri-o://1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994" gracePeriod=45
Jul 21 17:05:29.865141 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:29.865080    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:05:30.011833 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.011798    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-dshm\") pod \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") "
Jul 21 17:05:30.011833 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.011840    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-home\") pod \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") "
Jul 21 17:05:30.012342 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.011861    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l67p6\" (UniqueName: \"kubernetes.io/projected/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-kube-api-access-l67p6\") pod \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") "
Jul 21 17:05:30.012342 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.011877    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-model-cache\") pod \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") "
Jul 21 17:05:30.012342 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.012017    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tls-certs\") pod \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") "
Jul 21 17:05:30.012342 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.012087    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tmp-dir\") pod \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\" (UID: \"e8835e00-72bc-4262-a1fa-a32dfb69e0f5\") "
Jul 21 17:05:30.012342 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.012118    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "e8835e00-72bc-4262-a1fa-a32dfb69e0f5" (UID: "e8835e00-72bc-4262-a1fa-a32dfb69e0f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:05:30.012342 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.012133    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-home" (OuterVolumeSpecName: "home") pod "e8835e00-72bc-4262-a1fa-a32dfb69e0f5" (UID: "e8835e00-72bc-4262-a1fa-a32dfb69e0f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:05:30.012342 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.012265    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e8835e00-72bc-4262-a1fa-a32dfb69e0f5" (UID: "e8835e00-72bc-4262-a1fa-a32dfb69e0f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:05:30.012655 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.012392    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tmp-dir\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:30.012655 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.012414    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-home\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:30.012655 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.012428    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-model-cache\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:30.014319 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.014295    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e8835e00-72bc-4262-a1fa-a32dfb69e0f5" (UID: "e8835e00-72bc-4262-a1fa-a32dfb69e0f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:05:30.014571 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.014554    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-kube-api-access-l67p6" (OuterVolumeSpecName: "kube-api-access-l67p6") pod "e8835e00-72bc-4262-a1fa-a32dfb69e0f5" (UID: "e8835e00-72bc-4262-a1fa-a32dfb69e0f5"). InnerVolumeSpecName "kube-api-access-l67p6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:05:30.014571 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.014558    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-dshm" (OuterVolumeSpecName: "dshm") pod "e8835e00-72bc-4262-a1fa-a32dfb69e0f5" (UID: "e8835e00-72bc-4262-a1fa-a32dfb69e0f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:05:30.113752 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.113716    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l67p6\" (UniqueName: \"kubernetes.io/projected/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-kube-api-access-l67p6\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:30.113752 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.113747    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-tls-certs\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:30.113752 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.113755    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8835e00-72bc-4262-a1fa-a32dfb69e0f5-dshm\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:30.521838 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.521796    2580 generic.go:356] "Generic (PLEG): container finished" podID="e8835e00-72bc-4262-a1fa-a32dfb69e0f5" containerID="1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994" exitCode=0
Jul 21 17:05:30.521838 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.521844    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"
Jul 21 17:05:30.522071 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.521860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr" event={"ID":"e8835e00-72bc-4262-a1fa-a32dfb69e0f5","Type":"ContainerDied","Data":"1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994"}
Jul 21 17:05:30.522071 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.521891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr" event={"ID":"e8835e00-72bc-4262-a1fa-a32dfb69e0f5","Type":"ContainerDied","Data":"c401699e7a1cea7f5827c02f576f1729b5d53ca7eeba7ac7df7aac842eccfa8f"}
Jul 21 17:05:30.522071 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.521908    2580 scope.go:117] "RemoveContainer" containerID="1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994"
Jul 21 17:05:30.530412 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.530396    2580 scope.go:117] "RemoveContainer" containerID="1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994"
Jul 21 17:05:30.530664 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:05:30.530648    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994\": container with ID starting with 1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994 not found: ID does not exist" containerID="1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994"
Jul 21 17:05:30.530718 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.530674    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994"} err="failed to get container status \"1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994\": rpc error: code = NotFound desc = could not find container \"1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994\": container with ID starting with 1e43cec20d1cec569d7181b10ff792e3f54d04a9901584efa2608be633da8994 not found: ID does not exist"
Jul 21 17:05:30.545778 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.545753    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"]
Jul 21 17:05:30.555388 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.555365    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-981b1e/test-model-981b1e-kserve-79776b6dd9-vvjxr"]
Jul 21 17:05:30.939348 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:30.939311    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8835e00-72bc-4262-a1fa-a32dfb69e0f5" path="/var/lib/kubelet/pods/e8835e00-72bc-4262-a1fa-a32dfb69e0f5/volumes"
Jul 21 17:05:37.919602 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:37.919565    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"]
Jul 21 17:05:37.919984 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:37.919892    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8835e00-72bc-4262-a1fa-a32dfb69e0f5" containerName="main"
Jul 21 17:05:37.919984 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:37.919903    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8835e00-72bc-4262-a1fa-a32dfb69e0f5" containerName="main"
Jul 21 17:05:37.919984 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:37.919930    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02c5938d-e9c2-4108-91ad-773de8d04bb8" containerName="maas-api"
Jul 21 17:05:37.919984 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:37.919936    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="02c5938d-e9c2-4108-91ad-773de8d04bb8" containerName="maas-api"
Jul 21 17:05:37.920117 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:37.919990    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8835e00-72bc-4262-a1fa-a32dfb69e0f5" containerName="main"
Jul 21 17:05:37.920117 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:37.920003    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="02c5938d-e9c2-4108-91ad-773de8d04bb8" containerName="maas-api"
Jul 21 17:05:37.924357 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:37.924340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"
Jul 21 17:05:37.928937 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:37.928911    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 17:05:37.932373 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:37.932347    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"]
Jul 21 17:05:38.085371 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:38.085335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff071f1-60b1-449a-a320-ccc4a1f7cde6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n\" (UID: \"6ff071f1-60b1-449a-a320-ccc4a1f7cde6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"
Jul 21 17:05:38.186350 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:38.186254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff071f1-60b1-449a-a320-ccc4a1f7cde6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n\" (UID: \"6ff071f1-60b1-449a-a320-ccc4a1f7cde6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"
Jul 21 17:05:38.186914 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:38.186894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff071f1-60b1-449a-a320-ccc4a1f7cde6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n\" (UID: \"6ff071f1-60b1-449a-a320-ccc4a1f7cde6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"
Jul 21 17:05:38.236385 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:38.236355    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"
Jul 21 17:05:38.370672 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:38.370645    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"]
Jul 21 17:05:38.373317 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:05:38.373291    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff071f1_60b1_449a_a320_ccc4a1f7cde6.slice/crio-81cd8c84e11f7c8b5b4d253b5c3a74cb370c404fa419bafd076b60b48cd8240e WatchSource:0}: Error finding container 81cd8c84e11f7c8b5b4d253b5c3a74cb370c404fa419bafd076b60b48cd8240e: Status 404 returned error can't find the container with id 81cd8c84e11f7c8b5b4d253b5c3a74cb370c404fa419bafd076b60b48cd8240e
Jul 21 17:05:38.558891 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:38.558856    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n" event={"ID":"6ff071f1-60b1-449a-a320-ccc4a1f7cde6","Type":"ContainerStarted","Data":"81cd8c84e11f7c8b5b4d253b5c3a74cb370c404fa419bafd076b60b48cd8240e"}
Jul 21 17:05:39.570635 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:39.570604    2580 generic.go:356] "Generic (PLEG): container finished" podID="6ff071f1-60b1-449a-a320-ccc4a1f7cde6" containerID="6f5f3ae29c83f9b59495918cab921005805fbff97ac62da2dba86e3b9d7daf68" exitCode=0
Jul 21 17:05:39.571029 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:39.570695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n" event={"ID":"6ff071f1-60b1-449a-a320-ccc4a1f7cde6","Type":"ContainerDied","Data":"6f5f3ae29c83f9b59495918cab921005805fbff97ac62da2dba86e3b9d7daf68"}
Jul 21 17:05:40.702631 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:40.702603    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"
Jul 21 17:05:40.810760 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:40.810728    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff071f1-60b1-449a-a320-ccc4a1f7cde6-maas-api-service-ca\") pod \"6ff071f1-60b1-449a-a320-ccc4a1f7cde6\" (UID: \"6ff071f1-60b1-449a-a320-ccc4a1f7cde6\") "
Jul 21 17:05:40.811117 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:40.811088    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ff071f1-60b1-449a-a320-ccc4a1f7cde6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ff071f1-60b1-449a-a320-ccc4a1f7cde6" (UID: "6ff071f1-60b1-449a-a320-ccc4a1f7cde6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:05:40.912108 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:40.912066    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff071f1-60b1-449a-a320-ccc4a1f7cde6-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:41.580714 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:41.580676    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"
Jul 21 17:05:41.580888 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:41.580676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n" event={"ID":"6ff071f1-60b1-449a-a320-ccc4a1f7cde6","Type":"ContainerDied","Data":"81cd8c84e11f7c8b5b4d253b5c3a74cb370c404fa419bafd076b60b48cd8240e"}
Jul 21 17:05:41.580888 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:41.580827    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81cd8c84e11f7c8b5b4d253b5c3a74cb370c404fa419bafd076b60b48cd8240e"
Jul 21 17:05:48.377033 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.376948    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"]
Jul 21 17:05:48.377616 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.377234    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9" podUID="d7f34a70-d0c2-4501-a85e-056836f8e2b0" containerName="maas-api" containerID="cri-o://03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea" gracePeriod=30
Jul 21 17:05:48.820053 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.820026    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:05:48.882391 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.882360    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7f34a70-d0c2-4501-a85e-056836f8e2b0-openshift-service-ca\") pod \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") "
Jul 21 17:05:48.882561 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.882399    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7f34a70-d0c2-4501-a85e-056836f8e2b0-maas-api-tls\") pod \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") "
Jul 21 17:05:48.882561 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.882434    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x2bv\" (UniqueName: \"kubernetes.io/projected/d7f34a70-d0c2-4501-a85e-056836f8e2b0-kube-api-access-7x2bv\") pod \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\" (UID: \"d7f34a70-d0c2-4501-a85e-056836f8e2b0\") "
Jul 21 17:05:48.882785 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.882756    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7f34a70-d0c2-4501-a85e-056836f8e2b0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7f34a70-d0c2-4501-a85e-056836f8e2b0" (UID: "d7f34a70-d0c2-4501-a85e-056836f8e2b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:05:48.884643 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.884614    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7f34a70-d0c2-4501-a85e-056836f8e2b0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7f34a70-d0c2-4501-a85e-056836f8e2b0" (UID: "d7f34a70-d0c2-4501-a85e-056836f8e2b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:05:48.884742 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.884680    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7f34a70-d0c2-4501-a85e-056836f8e2b0-kube-api-access-7x2bv" (OuterVolumeSpecName: "kube-api-access-7x2bv") pod "d7f34a70-d0c2-4501-a85e-056836f8e2b0" (UID: "d7f34a70-d0c2-4501-a85e-056836f8e2b0"). InnerVolumeSpecName "kube-api-access-7x2bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:05:48.983708 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.983676    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7f34a70-d0c2-4501-a85e-056836f8e2b0-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:48.983708 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.983705    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7f34a70-d0c2-4501-a85e-056836f8e2b0-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:48.983924 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:48.983719    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x2bv\" (UniqueName: \"kubernetes.io/projected/d7f34a70-d0c2-4501-a85e-056836f8e2b0-kube-api-access-7x2bv\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:05:49.617528 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:49.617477    2580 generic.go:356] "Generic (PLEG): container finished" podID="d7f34a70-d0c2-4501-a85e-056836f8e2b0" containerID="03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea" exitCode=0
Jul 21 17:05:49.617963 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:49.617542    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"
Jul 21 17:05:49.617963 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:49.617568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9" event={"ID":"d7f34a70-d0c2-4501-a85e-056836f8e2b0","Type":"ContainerDied","Data":"03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea"}
Jul 21 17:05:49.617963 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:49.617613    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9" event={"ID":"d7f34a70-d0c2-4501-a85e-056836f8e2b0","Type":"ContainerDied","Data":"e817798f2a4cd6ea50e6ea5eda855fa840df5c667147a69aec166a720e13c403"}
Jul 21 17:05:49.617963 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:49.617639    2580 scope.go:117] "RemoveContainer" containerID="03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea"
Jul 21 17:05:49.626064 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:49.626043    2580 scope.go:117] "RemoveContainer" containerID="03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea"
Jul 21 17:05:49.626311 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:05:49.626290    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea\": container with ID starting with 03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea not found: ID does not exist" containerID="03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea"
Jul 21 17:05:49.626379 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:49.626317    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea"} err="failed to get container status \"03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea\": rpc error: code = NotFound desc = could not find container \"03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea\": container with ID starting with 03d9f17837612d8e712d73d0d72faaedadca1013232c5a4d47c6bc64e873c5ea not found: ID does not exist"
Jul 21 17:05:49.634951 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:49.634929    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"]
Jul 21 17:05:49.639406 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:49.639385    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fe1af7-5f589c79bf-dqqs9"]
Jul 21 17:05:50.939964 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:50.939930    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7f34a70-d0c2-4501-a85e-056836f8e2b0" path="/var/lib/kubelet/pods/d7f34a70-d0c2-4501-a85e-056836f8e2b0/volumes"
Jul 21 17:05:53.001826 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:52.998362    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"]
Jul 21 17:05:53.004021 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:53.003992    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-fe1af7-tsh5n"]
Jul 21 17:05:54.939262 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:05:54.939220    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ff071f1-60b1-449a-a320-ccc4a1f7cde6" path="/var/lib/kubelet/pods/6ff071f1-60b1-449a-a320-ccc4a1f7cde6/volumes"
Jul 21 17:06:03.200712 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.200683    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"]
Jul 21 17:06:03.201070 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.200967    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f34a70-d0c2-4501-a85e-056836f8e2b0" containerName="maas-api"
Jul 21 17:06:03.201070 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.200977    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f34a70-d0c2-4501-a85e-056836f8e2b0" containerName="maas-api"
Jul 21 17:06:03.201070 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.200997    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff071f1-60b1-449a-a320-ccc4a1f7cde6" containerName="revoke-keys"
Jul 21 17:06:03.201070 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.201003    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff071f1-60b1-449a-a320-ccc4a1f7cde6" containerName="revoke-keys"
Jul 21 17:06:03.201070 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.201055    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff071f1-60b1-449a-a320-ccc4a1f7cde6" containerName="revoke-keys"
Jul 21 17:06:03.201070 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.201063    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f34a70-d0c2-4501-a85e-056836f8e2b0" containerName="maas-api"
Jul 21 17:06:03.204109 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.204092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:03.207793 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.207768    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:06:03.207929 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.207773    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:06:03.207929 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.207775    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-927131-serving-cert\""
Jul 21 17:06:03.208613 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.208585    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9g9wv\""
Jul 21 17:06:03.227495 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.227473    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"]
Jul 21 17:06:03.309504 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.309474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-openshift-service-ca\") pod \"maas-api-e2e-api-a-927131-675889b7c6-j5qll\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:03.309650 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.309567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5vwd\" (UniqueName: \"kubernetes.io/projected/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-kube-api-access-z5vwd\") pod \"maas-api-e2e-api-a-927131-675889b7c6-j5qll\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:03.309650 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.309633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-maas-api-tls\") pod \"maas-api-e2e-api-a-927131-675889b7c6-j5qll\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:03.410479 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.410449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-openshift-service-ca\") pod \"maas-api-e2e-api-a-927131-675889b7c6-j5qll\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:03.410479 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.410480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5vwd\" (UniqueName: \"kubernetes.io/projected/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-kube-api-access-z5vwd\") pod \"maas-api-e2e-api-a-927131-675889b7c6-j5qll\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:03.410678 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.410507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-maas-api-tls\") pod \"maas-api-e2e-api-a-927131-675889b7c6-j5qll\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:03.410678 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:06:03.410626    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-927131-serving-cert: secret "maas-api-e2e-api-a-927131-serving-cert" not found
Jul 21 17:06:03.410751 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:06:03.410688    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-maas-api-tls podName:a3896ba7-a0be-482e-9d93-36cd06dc8e5c nodeName:}" failed. No retries permitted until 2026-07-21 17:06:03.910671326 +0000 UTC m=+2259.410032548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-maas-api-tls") pod "maas-api-e2e-api-a-927131-675889b7c6-j5qll" (UID: "a3896ba7-a0be-482e-9d93-36cd06dc8e5c") : secret "maas-api-e2e-api-a-927131-serving-cert" not found
Jul 21 17:06:03.411074 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.411057    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-openshift-service-ca\") pod \"maas-api-e2e-api-a-927131-675889b7c6-j5qll\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:03.429164 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.429145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5vwd\" (UniqueName: \"kubernetes.io/projected/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-kube-api-access-z5vwd\") pod \"maas-api-e2e-api-a-927131-675889b7c6-j5qll\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:03.913858 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.913828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-maas-api-tls\") pod \"maas-api-e2e-api-a-927131-675889b7c6-j5qll\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:03.916344 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:03.916327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-maas-api-tls\") pod \"maas-api-e2e-api-a-927131-675889b7c6-j5qll\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:04.114698 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:04.114665    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:04.248114 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:04.248092    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"]
Jul 21 17:06:04.250198 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:06:04.250171    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3896ba7_a0be_482e_9d93_36cd06dc8e5c.slice/crio-ef5d5f91ceab8d4b674e3b00836dba5d00a06e534858fa7a8d98ca08a949173b WatchSource:0}: Error finding container ef5d5f91ceab8d4b674e3b00836dba5d00a06e534858fa7a8d98ca08a949173b: Status 404 returned error can't find the container with id ef5d5f91ceab8d4b674e3b00836dba5d00a06e534858fa7a8d98ca08a949173b
Jul 21 17:06:04.684947 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:04.684919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll" event={"ID":"a3896ba7-a0be-482e-9d93-36cd06dc8e5c","Type":"ContainerStarted","Data":"ef5d5f91ceab8d4b674e3b00836dba5d00a06e534858fa7a8d98ca08a949173b"}
Jul 21 17:06:05.690373 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:05.690338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll" event={"ID":"a3896ba7-a0be-482e-9d93-36cd06dc8e5c","Type":"ContainerStarted","Data":"8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8"}
Jul 21 17:06:05.690786 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:05.690471    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:05.708653 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:05.708602    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll" podStartSLOduration=2.313541303 podStartE2EDuration="2.708590389s" totalImagesPullingTime="395.049086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:06:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:06:05.707382023 +0000 UTC m=+2261.206743269" watchObservedRunningTime="2026-07-21 17:06:05.708590389 +0000 UTC m=+2261.207951633"
Jul 21 17:06:11.096294 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.096258    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"]
Jul 21 17:06:11.099628 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.099609    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.102672 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.102652    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-6f8407-serving-cert\""
Jul 21 17:06:11.117264 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.117232    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"]
Jul 21 17:06:11.178006 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.177971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9edcb5db-70cf-46db-bbb8-a395abd0dc02-maas-api-tls\") pod \"maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.178006 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.178011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6ssh\" (UniqueName: \"kubernetes.io/projected/9edcb5db-70cf-46db-bbb8-a395abd0dc02-kube-api-access-p6ssh\") pod \"maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.178237 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.178134    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9edcb5db-70cf-46db-bbb8-a395abd0dc02-openshift-service-ca\") pod \"maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.279591 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.279551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9edcb5db-70cf-46db-bbb8-a395abd0dc02-maas-api-tls\") pod \"maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.279591 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.279596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6ssh\" (UniqueName: \"kubernetes.io/projected/9edcb5db-70cf-46db-bbb8-a395abd0dc02-kube-api-access-p6ssh\") pod \"maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.279823 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.279667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9edcb5db-70cf-46db-bbb8-a395abd0dc02-openshift-service-ca\") pod \"maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.280312 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.280252    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9edcb5db-70cf-46db-bbb8-a395abd0dc02-openshift-service-ca\") pod \"maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.282123 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.282101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9edcb5db-70cf-46db-bbb8-a395abd0dc02-maas-api-tls\") pod \"maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.296115 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.296085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6ssh\" (UniqueName: \"kubernetes.io/projected/9edcb5db-70cf-46db-bbb8-a395abd0dc02-kube-api-access-p6ssh\") pod \"maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.409158 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.409128    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:11.578147 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.578114    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"]
Jul 21 17:06:11.582210 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:06:11.582180    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9edcb5db_70cf_46db_bbb8_a395abd0dc02.slice/crio-6621d2370ba5849619221ecf15fea7c5cdc22f5da52932358d5f94569fdad776 WatchSource:0}: Error finding container 6621d2370ba5849619221ecf15fea7c5cdc22f5da52932358d5f94569fdad776: Status 404 returned error can't find the container with id 6621d2370ba5849619221ecf15fea7c5cdc22f5da52932358d5f94569fdad776
Jul 21 17:06:11.700786 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.700707    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:11.723382 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:11.723346    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz" event={"ID":"9edcb5db-70cf-46db-bbb8-a395abd0dc02","Type":"ContainerStarted","Data":"6621d2370ba5849619221ecf15fea7c5cdc22f5da52932358d5f94569fdad776"}
Jul 21 17:06:12.729588 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:12.729542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz" event={"ID":"9edcb5db-70cf-46db-bbb8-a395abd0dc02","Type":"ContainerStarted","Data":"82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e"}
Jul 21 17:06:12.730052 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:12.729613    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:12.765776 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:12.765713    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz" podStartSLOduration=1.359382217 podStartE2EDuration="1.765688875s" totalImagesPullingTime="406.306658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:06:12.763727441 +0000 UTC m=+2268.263088703" watchObservedRunningTime="2026-07-21 17:06:12.765688875 +0000 UTC m=+2268.265050119"
Jul 21 17:06:18.739696 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:18.739661    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:06:23.398400 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:23.398361    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"]
Jul 21 17:06:23.401820 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:23.401803    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"
Jul 21 17:06:23.405306 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:23.405284    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 17:06:23.416933 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:23.416913    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"]
Jul 21 17:06:23.586957 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:23.586923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d51eb8a-73b7-4890-b98c-f88af6b8b03e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-927131-c85qq\" (UID: \"4d51eb8a-73b7-4890-b98c-f88af6b8b03e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"
Jul 21 17:06:23.688194 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:23.688164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d51eb8a-73b7-4890-b98c-f88af6b8b03e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-927131-c85qq\" (UID: \"4d51eb8a-73b7-4890-b98c-f88af6b8b03e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"
Jul 21 17:06:23.688850 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:23.688831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d51eb8a-73b7-4890-b98c-f88af6b8b03e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-927131-c85qq\" (UID: \"4d51eb8a-73b7-4890-b98c-f88af6b8b03e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"
Jul 21 17:06:23.712038 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:23.712019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"
Jul 21 17:06:23.846061 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:23.845923    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"]
Jul 21 17:06:23.848816 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:06:23.848789    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d51eb8a_73b7_4890_b98c_f88af6b8b03e.slice/crio-faa963eb2e840d15088df7de6c89e88fde63b69d51500a38ee7ce7a42cd8eea4 WatchSource:0}: Error finding container faa963eb2e840d15088df7de6c89e88fde63b69d51500a38ee7ce7a42cd8eea4: Status 404 returned error can't find the container with id faa963eb2e840d15088df7de6c89e88fde63b69d51500a38ee7ce7a42cd8eea4
Jul 21 17:06:24.786393 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:24.786307    2580 generic.go:356] "Generic (PLEG): container finished" podID="4d51eb8a-73b7-4890-b98c-f88af6b8b03e" containerID="f0c8f0e16752fd46a581438751eceecce285d96928e029c3697c73b950d8c90e" exitCode=0
Jul 21 17:06:24.786785 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:24.786397    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq" event={"ID":"4d51eb8a-73b7-4890-b98c-f88af6b8b03e","Type":"ContainerDied","Data":"f0c8f0e16752fd46a581438751eceecce285d96928e029c3697c73b950d8c90e"}
Jul 21 17:06:24.786785 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:24.786447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq" event={"ID":"4d51eb8a-73b7-4890-b98c-f88af6b8b03e","Type":"ContainerStarted","Data":"faa963eb2e840d15088df7de6c89e88fde63b69d51500a38ee7ce7a42cd8eea4"}
Jul 21 17:06:25.124172 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:25.124091    2580 scope.go:117] "RemoveContainer" containerID="f44079f32b788d749eec4960d197f196cebe8357b69d82db4e293ff1d6d22fa1"
Jul 21 17:06:25.923492 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:25.923465    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"
Jul 21 17:06:26.109872 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:26.109792    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d51eb8a-73b7-4890-b98c-f88af6b8b03e-maas-api-service-ca\") pod \"4d51eb8a-73b7-4890-b98c-f88af6b8b03e\" (UID: \"4d51eb8a-73b7-4890-b98c-f88af6b8b03e\") "
Jul 21 17:06:26.110178 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:26.110153    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d51eb8a-73b7-4890-b98c-f88af6b8b03e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d51eb8a-73b7-4890-b98c-f88af6b8b03e" (UID: "4d51eb8a-73b7-4890-b98c-f88af6b8b03e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:06:26.210864 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:26.210837    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d51eb8a-73b7-4890-b98c-f88af6b8b03e-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:06:26.797275 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:26.797246    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"
Jul 21 17:06:26.797275 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:26.797264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq" event={"ID":"4d51eb8a-73b7-4890-b98c-f88af6b8b03e","Type":"ContainerDied","Data":"faa963eb2e840d15088df7de6c89e88fde63b69d51500a38ee7ce7a42cd8eea4"}
Jul 21 17:06:26.797476 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:26.797305    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="faa963eb2e840d15088df7de6c89e88fde63b69d51500a38ee7ce7a42cd8eea4"
Jul 21 17:06:38.755654 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:38.755624    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"]
Jul 21 17:06:38.756124 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:38.755858    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll" podUID="a3896ba7-a0be-482e-9d93-36cd06dc8e5c" containerName="maas-api" containerID="cri-o://8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8" gracePeriod=30
Jul 21 17:06:39.002179 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.002155    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:39.118689 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.118601    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-maas-api-tls\") pod \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") "
Jul 21 17:06:39.118689 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.118649    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-openshift-service-ca\") pod \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") "
Jul 21 17:06:39.118912 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.118701    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5vwd\" (UniqueName: \"kubernetes.io/projected/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-kube-api-access-z5vwd\") pod \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\" (UID: \"a3896ba7-a0be-482e-9d93-36cd06dc8e5c\") "
Jul 21 17:06:39.119032 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.119004    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a3896ba7-a0be-482e-9d93-36cd06dc8e5c" (UID: "a3896ba7-a0be-482e-9d93-36cd06dc8e5c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:06:39.120907 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.120869    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-kube-api-access-z5vwd" (OuterVolumeSpecName: "kube-api-access-z5vwd") pod "a3896ba7-a0be-482e-9d93-36cd06dc8e5c" (UID: "a3896ba7-a0be-482e-9d93-36cd06dc8e5c"). InnerVolumeSpecName "kube-api-access-z5vwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:06:39.120907 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.120894    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a3896ba7-a0be-482e-9d93-36cd06dc8e5c" (UID: "a3896ba7-a0be-482e-9d93-36cd06dc8e5c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:06:39.220191 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.220152    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5vwd\" (UniqueName: \"kubernetes.io/projected/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-kube-api-access-z5vwd\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:06:39.220191 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.220184    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:06:39.220191 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.220193    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3896ba7-a0be-482e-9d93-36cd06dc8e5c-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:06:39.853633 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.853597    2580 generic.go:356] "Generic (PLEG): container finished" podID="a3896ba7-a0be-482e-9d93-36cd06dc8e5c" containerID="8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8" exitCode=0
Jul 21 17:06:39.854132 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.853648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll" event={"ID":"a3896ba7-a0be-482e-9d93-36cd06dc8e5c","Type":"ContainerDied","Data":"8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8"}
Jul 21 17:06:39.854132 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.853653    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"
Jul 21 17:06:39.854132 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.853695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll" event={"ID":"a3896ba7-a0be-482e-9d93-36cd06dc8e5c","Type":"ContainerDied","Data":"ef5d5f91ceab8d4b674e3b00836dba5d00a06e534858fa7a8d98ca08a949173b"}
Jul 21 17:06:39.854132 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.853717    2580 scope.go:117] "RemoveContainer" containerID="8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8"
Jul 21 17:06:39.863075 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.863055    2580 scope.go:117] "RemoveContainer" containerID="8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8"
Jul 21 17:06:39.863338 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:06:39.863317    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8\": container with ID starting with 8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8 not found: ID does not exist" containerID="8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8"
Jul 21 17:06:39.863396 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.863345    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8"} err="failed to get container status \"8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8\": rpc error: code = NotFound desc = could not find container \"8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8\": container with ID starting with 8c420bada803ad1c0c63518af1e95d4836a3e663ddd9987d02caa7edb7c90cd8 not found: ID does not exist"
Jul 21 17:06:39.877444 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.877414    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"]
Jul 21 17:06:39.879966 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:39.879942    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-927131-675889b7c6-j5qll"]
Jul 21 17:06:40.939403 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:40.939370    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3896ba7-a0be-482e-9d93-36cd06dc8e5c" path="/var/lib/kubelet/pods/a3896ba7-a0be-482e-9d93-36cd06dc8e5c/volumes"
Jul 21 17:06:43.505711 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:43.505675    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"]
Jul 21 17:06:43.512203 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:43.512172    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-927131-c85qq"]
Jul 21 17:06:44.940850 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:44.940815    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d51eb8a-73b7-4890-b98c-f88af6b8b03e" path="/var/lib/kubelet/pods/4d51eb8a-73b7-4890-b98c-f88af6b8b03e/volumes"
Jul 21 17:06:50.234993 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.234949    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"]
Jul 21 17:06:50.235550 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.235394    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3896ba7-a0be-482e-9d93-36cd06dc8e5c" containerName="maas-api"
Jul 21 17:06:50.235550 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.235411    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3896ba7-a0be-482e-9d93-36cd06dc8e5c" containerName="maas-api"
Jul 21 17:06:50.235550 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.235437    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d51eb8a-73b7-4890-b98c-f88af6b8b03e" containerName="revoke-keys"
Jul 21 17:06:50.235550 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.235446    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d51eb8a-73b7-4890-b98c-f88af6b8b03e" containerName="revoke-keys"
Jul 21 17:06:50.235550 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.235536    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3896ba7-a0be-482e-9d93-36cd06dc8e5c" containerName="maas-api"
Jul 21 17:06:50.235550 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.235551    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d51eb8a-73b7-4890-b98c-f88af6b8b03e" containerName="revoke-keys"
Jul 21 17:06:50.238868 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.238833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"
Jul 21 17:06:50.241582 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.241556    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 17:06:50.246978 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.246942    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"]
Jul 21 17:06:50.316351 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.316318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7\" (UID: \"d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"
Jul 21 17:06:50.417671 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.417637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7\" (UID: \"d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"
Jul 21 17:06:50.418249 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.418229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7\" (UID: \"d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"
Jul 21 17:06:50.549431 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.549337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"
Jul 21 17:06:50.687639 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.687607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"]
Jul 21 17:06:50.691370 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:06:50.691339    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8e5c17f_34c6_4c7d_aa5c_adbd947ad2db.slice/crio-8727cc9ab32485d0587b39dabe48530c0e2a31cbfe9294012cb2cccacea34c66 WatchSource:0}: Error finding container 8727cc9ab32485d0587b39dabe48530c0e2a31cbfe9294012cb2cccacea34c66: Status 404 returned error can't find the container with id 8727cc9ab32485d0587b39dabe48530c0e2a31cbfe9294012cb2cccacea34c66
Jul 21 17:06:50.904122 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:50.904083    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7" event={"ID":"d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db","Type":"ContainerStarted","Data":"8727cc9ab32485d0587b39dabe48530c0e2a31cbfe9294012cb2cccacea34c66"}
Jul 21 17:06:51.910716 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:51.910680    2580 generic.go:356] "Generic (PLEG): container finished" podID="d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db" containerID="bb8e24c2a0401a71e231d23ae0024cd1571763a4dc601a49598476d7fe8cb8fb" exitCode=0
Jul 21 17:06:51.911195 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:51.910766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7" event={"ID":"d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db","Type":"ContainerDied","Data":"bb8e24c2a0401a71e231d23ae0024cd1571763a4dc601a49598476d7fe8cb8fb"}
Jul 21 17:06:53.046740 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:53.046703    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"
Jul 21 17:06:53.140426 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:53.140392    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db-maas-api-service-ca\") pod \"d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db\" (UID: \"d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db\") "
Jul 21 17:06:53.140833 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:53.140812    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db" (UID: "d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:06:53.241289 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:53.241195    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:06:53.921648 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:53.921568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7" event={"ID":"d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db","Type":"ContainerDied","Data":"8727cc9ab32485d0587b39dabe48530c0e2a31cbfe9294012cb2cccacea34c66"}
Jul 21 17:06:53.921648 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:53.921630    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8727cc9ab32485d0587b39dabe48530c0e2a31cbfe9294012cb2cccacea34c66"
Jul 21 17:06:53.921648 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:06:53.921580    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"
Jul 21 17:07:06.100558 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.100503    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"]
Jul 21 17:07:06.101103 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.100749    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz" podUID="9edcb5db-70cf-46db-bbb8-a395abd0dc02" containerName="maas-api" containerID="cri-o://82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e" gracePeriod=30
Jul 21 17:07:06.347853 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.347827    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:07:06.461901 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.461869    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6ssh\" (UniqueName: \"kubernetes.io/projected/9edcb5db-70cf-46db-bbb8-a395abd0dc02-kube-api-access-p6ssh\") pod \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") "
Jul 21 17:07:06.462078 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.461966    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9edcb5db-70cf-46db-bbb8-a395abd0dc02-openshift-service-ca\") pod \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") "
Jul 21 17:07:06.462078 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.462049    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9edcb5db-70cf-46db-bbb8-a395abd0dc02-maas-api-tls\") pod \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\" (UID: \"9edcb5db-70cf-46db-bbb8-a395abd0dc02\") "
Jul 21 17:07:06.462395 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.462360    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9edcb5db-70cf-46db-bbb8-a395abd0dc02-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9edcb5db-70cf-46db-bbb8-a395abd0dc02" (UID: "9edcb5db-70cf-46db-bbb8-a395abd0dc02"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:07:06.464290 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.464264    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9edcb5db-70cf-46db-bbb8-a395abd0dc02-kube-api-access-p6ssh" (OuterVolumeSpecName: "kube-api-access-p6ssh") pod "9edcb5db-70cf-46db-bbb8-a395abd0dc02" (UID: "9edcb5db-70cf-46db-bbb8-a395abd0dc02"). InnerVolumeSpecName "kube-api-access-p6ssh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:07:06.464290 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.464270    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9edcb5db-70cf-46db-bbb8-a395abd0dc02-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9edcb5db-70cf-46db-bbb8-a395abd0dc02" (UID: "9edcb5db-70cf-46db-bbb8-a395abd0dc02"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:07:06.563004 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.562964    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9edcb5db-70cf-46db-bbb8-a395abd0dc02-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:07:06.563004 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.562996    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9edcb5db-70cf-46db-bbb8-a395abd0dc02-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:07:06.563004 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.563006    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6ssh\" (UniqueName: \"kubernetes.io/projected/9edcb5db-70cf-46db-bbb8-a395abd0dc02-kube-api-access-p6ssh\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:07:06.981259 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.981226    2580 generic.go:356] "Generic (PLEG): container finished" podID="9edcb5db-70cf-46db-bbb8-a395abd0dc02" containerID="82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e" exitCode=0
Jul 21 17:07:06.981259 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.981260    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"
Jul 21 17:07:06.981552 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.981304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz" event={"ID":"9edcb5db-70cf-46db-bbb8-a395abd0dc02","Type":"ContainerDied","Data":"82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e"}
Jul 21 17:07:06.981552 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.981358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz" event={"ID":"9edcb5db-70cf-46db-bbb8-a395abd0dc02","Type":"ContainerDied","Data":"6621d2370ba5849619221ecf15fea7c5cdc22f5da52932358d5f94569fdad776"}
Jul 21 17:07:06.981552 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.981388    2580 scope.go:117] "RemoveContainer" containerID="82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e"
Jul 21 17:07:06.989780 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.989762    2580 scope.go:117] "RemoveContainer" containerID="82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e"
Jul 21 17:07:06.990061 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:07:06.990044    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e\": container with ID starting with 82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e not found: ID does not exist" containerID="82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e"
Jul 21 17:07:06.990114 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.990071    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e"} err="failed to get container status \"82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e\": rpc error: code = NotFound desc = could not find container \"82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e\": container with ID starting with 82c9c828cd10b226eb373cbe84427c258e1d7c7ab0dcec3878dd3317c7f8d77e not found: ID does not exist"
Jul 21 17:07:06.999577 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:06.999550    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"]
Jul 21 17:07:07.003757 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:07.003736    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8407-575b88c4c7-qr8qz"]
Jul 21 17:07:08.939630 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:08.939599    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9edcb5db-70cf-46db-bbb8-a395abd0dc02" path="/var/lib/kubelet/pods/9edcb5db-70cf-46db-bbb8-a395abd0dc02/volumes"
Jul 21 17:07:10.328911 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:10.328877    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"]
Jul 21 17:07:10.331893 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:10.331868    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8407-ql8c7"]
Jul 21 17:07:10.939531 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:10.939480    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db" path="/var/lib/kubelet/pods/d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db/volumes"
Jul 21 17:07:20.468361 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.468288    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"]
Jul 21 17:07:20.468749 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.468677    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db" containerName="revoke-keys"
Jul 21 17:07:20.468749 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.468692    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db" containerName="revoke-keys"
Jul 21 17:07:20.468749 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.468711    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9edcb5db-70cf-46db-bbb8-a395abd0dc02" containerName="maas-api"
Jul 21 17:07:20.468749 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.468717    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9edcb5db-70cf-46db-bbb8-a395abd0dc02" containerName="maas-api"
Jul 21 17:07:20.468886 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.468763    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8e5c17f-34c6-4c7d-aa5c-adbd947ad2db" containerName="revoke-keys"
Jul 21 17:07:20.468886 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.468772    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9edcb5db-70cf-46db-bbb8-a395abd0dc02" containerName="maas-api"
Jul 21 17:07:20.472913 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.472892    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:20.476372 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.476348    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9g9wv\""
Jul 21 17:07:20.476499 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.476348    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:07:20.476858 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.476843    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-5b0b89-serving-cert\""
Jul 21 17:07:20.476949 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.476846    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:07:20.484437 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.484413    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"]
Jul 21 17:07:20.576401 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.576368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:20.576605 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.576428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kff4\" (UniqueName: \"kubernetes.io/projected/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-kube-api-access-6kff4\") pod \"maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:20.576605 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.576466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-maas-api-tls\") pod \"maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:20.676988 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.676955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:20.677177 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.677015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kff4\" (UniqueName: \"kubernetes.io/projected/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-kube-api-access-6kff4\") pod \"maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:20.677177 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.677037    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-maas-api-tls\") pod \"maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:20.677177 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:07:20.677121    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-serving-cert: secret "maas-api-e2e-shared-a-5b0b89-serving-cert" not found
Jul 21 17:07:20.677177 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:07:20.677180    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-maas-api-tls podName:c0e36ac7-7518-4cd7-bfb3-2d9f63598436 nodeName:}" failed. No retries permitted until 2026-07-21 17:07:21.177164593 +0000 UTC m=+2336.676525815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-maas-api-tls") pod "maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9" (UID: "c0e36ac7-7518-4cd7-bfb3-2d9f63598436") : secret "maas-api-e2e-shared-a-5b0b89-serving-cert" not found
Jul 21 17:07:20.677589 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.677571    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:20.688962 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:20.688944    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kff4\" (UniqueName: \"kubernetes.io/projected/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-kube-api-access-6kff4\") pod \"maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:21.182869 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:21.182837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-maas-api-tls\") pod \"maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:21.185405 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:21.185375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-maas-api-tls\") pod \"maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:21.384020 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:21.383979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:21.521775 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:21.521690    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"]
Jul 21 17:07:21.524060 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:07:21.524034    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e36ac7_7518_4cd7_bfb3_2d9f63598436.slice/crio-a03c6dd03eaa7ade7409c86094cc910fabf29faa2d3f26ae25e57894c1208849 WatchSource:0}: Error finding container a03c6dd03eaa7ade7409c86094cc910fabf29faa2d3f26ae25e57894c1208849: Status 404 returned error can't find the container with id a03c6dd03eaa7ade7409c86094cc910fabf29faa2d3f26ae25e57894c1208849
Jul 21 17:07:22.048188 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:22.048155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9" event={"ID":"c0e36ac7-7518-4cd7-bfb3-2d9f63598436","Type":"ContainerStarted","Data":"461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb"}
Jul 21 17:07:22.048373 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:22.048245    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:22.048373 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:22.048267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9" event={"ID":"c0e36ac7-7518-4cd7-bfb3-2d9f63598436","Type":"ContainerStarted","Data":"a03c6dd03eaa7ade7409c86094cc910fabf29faa2d3f26ae25e57894c1208849"}
Jul 21 17:07:22.073968 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:22.073917    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9" podStartSLOduration=1.722133503 podStartE2EDuration="2.073901077s" totalImagesPullingTime="351.767574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:07:22.073085636 +0000 UTC m=+2337.572446883" watchObservedRunningTime="2026-07-21 17:07:22.073901077 +0000 UTC m=+2337.573262320"
Jul 21 17:07:25.156488 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:25.156459    2580 scope.go:117] "RemoveContainer" containerID="7a1642e74019a2d6dcd971bd7464ed360ac5578b1aef1aedbc7efdb03d43beff"
Jul 21 17:07:28.057547 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.057507    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:07:28.495262 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.495224    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"]
Jul 21 17:07:28.498918 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.498894    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.501940 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.501922    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-e17cb5-serving-cert\""
Jul 21 17:07:28.512821 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.512792    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"]
Jul 21 17:07:28.547018 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.546991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a31ed9-5e5c-41ad-a605-9de6328f17e5-maas-api-tls\") pod \"maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.547154 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.547052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m72md\" (UniqueName: \"kubernetes.io/projected/20a31ed9-5e5c-41ad-a605-9de6328f17e5-kube-api-access-m72md\") pod \"maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.547154 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.547122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a31ed9-5e5c-41ad-a605-9de6328f17e5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.648107 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.648073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a31ed9-5e5c-41ad-a605-9de6328f17e5-maas-api-tls\") pod \"maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.648255 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.648137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m72md\" (UniqueName: \"kubernetes.io/projected/20a31ed9-5e5c-41ad-a605-9de6328f17e5-kube-api-access-m72md\") pod \"maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.648255 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.648170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a31ed9-5e5c-41ad-a605-9de6328f17e5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.648808 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.648786    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a31ed9-5e5c-41ad-a605-9de6328f17e5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.651135 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.651102    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a31ed9-5e5c-41ad-a605-9de6328f17e5-maas-api-tls\") pod \"maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.659759 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.659703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m72md\" (UniqueName: \"kubernetes.io/projected/20a31ed9-5e5c-41ad-a605-9de6328f17e5-kube-api-access-m72md\") pod \"maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.816874 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.816843    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:28.954396 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:28.954373    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"]
Jul 21 17:07:28.956872 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:07:28.956843    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a31ed9_5e5c_41ad_a605_9de6328f17e5.slice/crio-dd88258462c1f4eeffba2027b42122a8ad95f4a8dbd0a37a984d1d96452a039e WatchSource:0}: Error finding container dd88258462c1f4eeffba2027b42122a8ad95f4a8dbd0a37a984d1d96452a039e: Status 404 returned error can't find the container with id dd88258462c1f4eeffba2027b42122a8ad95f4a8dbd0a37a984d1d96452a039e
Jul 21 17:07:29.079233 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:29.079204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6" event={"ID":"20a31ed9-5e5c-41ad-a605-9de6328f17e5","Type":"ContainerStarted","Data":"dd88258462c1f4eeffba2027b42122a8ad95f4a8dbd0a37a984d1d96452a039e"}
Jul 21 17:07:30.084426 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:30.084393    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6" event={"ID":"20a31ed9-5e5c-41ad-a605-9de6328f17e5","Type":"ContainerStarted","Data":"17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb"}
Jul 21 17:07:30.084904 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:30.084459    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:30.101637 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:30.101595    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6" podStartSLOduration=1.6911496480000001 podStartE2EDuration="2.101580597s" totalImagesPullingTime="410.430949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:07:30.101061079 +0000 UTC m=+2345.600422352" watchObservedRunningTime="2026-07-21 17:07:30.101580597 +0000 UTC m=+2345.600941840"
Jul 21 17:07:36.093496 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:36.093468    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:07:42.068453 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.068424    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"]
Jul 21 17:07:42.072152 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.072132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.076348 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.076323    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5b0b89\"/\"kube-root-ca.crt\""
Jul 21 17:07:42.076348 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.076343    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5b0b89\"/\"auth-test-model-5b0b89-kserve-self-signed-certs\""
Jul 21 17:07:42.076691 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.076676    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5b0b89\"/\"default-dockercfg-q9b56\""
Jul 21 17:07:42.076785 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.076707    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5b0b89\"/\"openshift-service-ca.crt\""
Jul 21 17:07:42.082137 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.082114    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"]
Jul 21 17:07:42.159252 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.159221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-home\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.159252 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.159256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-tmp-dir\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.159490 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.159368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db2ed92c-6453-4cc6-a686-960dd789a40f-tls-certs\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.159490 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.159405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqb7r\" (UniqueName: \"kubernetes.io/projected/db2ed92c-6453-4cc6-a686-960dd789a40f-kube-api-access-fqb7r\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.159490 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.159455    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-dshm\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.159490 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.159477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-model-cache\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.260865 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.260836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db2ed92c-6453-4cc6-a686-960dd789a40f-tls-certs\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.261035 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.260873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqb7r\" (UniqueName: \"kubernetes.io/projected/db2ed92c-6453-4cc6-a686-960dd789a40f-kube-api-access-fqb7r\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.261035 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.260898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-dshm\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.261035 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.260916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-model-cache\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.261035 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.260938    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-home\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.261035 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.260958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-tmp-dir\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.261358 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.261337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-model-cache\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.261421 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.261346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-tmp-dir\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.261421 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.261375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-home\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.263431 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.263411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-dshm\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.263636 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.263619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db2ed92c-6453-4cc6-a686-960dd789a40f-tls-certs\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.270003 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.269975    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqb7r\" (UniqueName: \"kubernetes.io/projected/db2ed92c-6453-4cc6-a686-960dd789a40f-kube-api-access-fqb7r\") pod \"auth-test-model-5b0b89-kserve-f64874475-qbzdh\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") " pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.381548 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.381445    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:42.528821 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:42.528796    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"]
Jul 21 17:07:42.531350 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:07:42.531316    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb2ed92c_6453_4cc6_a686_960dd789a40f.slice/crio-86fe57bf646e651fab420413b002a21690b8ecd2c530eabc09b0ef3eb1cb7946 WatchSource:0}: Error finding container 86fe57bf646e651fab420413b002a21690b8ecd2c530eabc09b0ef3eb1cb7946: Status 404 returned error can't find the container with id 86fe57bf646e651fab420413b002a21690b8ecd2c530eabc09b0ef3eb1cb7946
Jul 21 17:07:43.143887 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:43.143850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh" event={"ID":"db2ed92c-6453-4cc6-a686-960dd789a40f","Type":"ContainerStarted","Data":"97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea"}
Jul 21 17:07:43.143887 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:43.143887    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh" event={"ID":"db2ed92c-6453-4cc6-a686-960dd789a40f","Type":"ContainerStarted","Data":"86fe57bf646e651fab420413b002a21690b8ecd2c530eabc09b0ef3eb1cb7946"}
Jul 21 17:07:43.162804 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:43.162763    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh" podStartSLOduration=1.162748527 podStartE2EDuration="1.162748527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:07:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:07:43.161354368 +0000 UTC m=+2358.660715612" watchObservedRunningTime="2026-07-21 17:07:43.162748527 +0000 UTC m=+2358.662109782"
Jul 21 17:07:52.381902 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:52.381869    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:52.382365 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:52.381918    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:52.394476 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:52.394444    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:07:53.208529 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:07:53.208490    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:10:24.466080 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:24.466044    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"]
Jul 21 17:10:39.486420 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.486355    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh" podUID="db2ed92c-6453-4cc6-a686-960dd789a40f" containerName="main" containerID="cri-o://97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea" gracePeriod=45
Jul 21 17:10:39.720954 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.720931    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:10:39.778193 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778097    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-home\") pod \"db2ed92c-6453-4cc6-a686-960dd789a40f\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") "
Jul 21 17:10:39.778193 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778143    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqb7r\" (UniqueName: \"kubernetes.io/projected/db2ed92c-6453-4cc6-a686-960dd789a40f-kube-api-access-fqb7r\") pod \"db2ed92c-6453-4cc6-a686-960dd789a40f\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") "
Jul 21 17:10:39.778433 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778202    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-model-cache\") pod \"db2ed92c-6453-4cc6-a686-960dd789a40f\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") "
Jul 21 17:10:39.778433 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778258    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db2ed92c-6453-4cc6-a686-960dd789a40f-tls-certs\") pod \"db2ed92c-6453-4cc6-a686-960dd789a40f\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") "
Jul 21 17:10:39.778433 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778312    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-tmp-dir\") pod \"db2ed92c-6453-4cc6-a686-960dd789a40f\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") "
Jul 21 17:10:39.778433 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778348    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-dshm\") pod \"db2ed92c-6453-4cc6-a686-960dd789a40f\" (UID: \"db2ed92c-6453-4cc6-a686-960dd789a40f\") "
Jul 21 17:10:39.778662 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778444    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-home" (OuterVolumeSpecName: "home") pod "db2ed92c-6453-4cc6-a686-960dd789a40f" (UID: "db2ed92c-6453-4cc6-a686-960dd789a40f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:10:39.778662 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778493    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-model-cache" (OuterVolumeSpecName: "model-cache") pod "db2ed92c-6453-4cc6-a686-960dd789a40f" (UID: "db2ed92c-6453-4cc6-a686-960dd789a40f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:10:39.778662 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778569    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db2ed92c-6453-4cc6-a686-960dd789a40f" (UID: "db2ed92c-6453-4cc6-a686-960dd789a40f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:10:39.778819 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778757    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-tmp-dir\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:10:39.778819 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778778    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-home\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:10:39.778819 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.778791    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-model-cache\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:10:39.780645 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.780617    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-dshm" (OuterVolumeSpecName: "dshm") pod "db2ed92c-6453-4cc6-a686-960dd789a40f" (UID: "db2ed92c-6453-4cc6-a686-960dd789a40f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:10:39.780645 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.780634    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db2ed92c-6453-4cc6-a686-960dd789a40f-kube-api-access-fqb7r" (OuterVolumeSpecName: "kube-api-access-fqb7r") pod "db2ed92c-6453-4cc6-a686-960dd789a40f" (UID: "db2ed92c-6453-4cc6-a686-960dd789a40f"). InnerVolumeSpecName "kube-api-access-fqb7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:10:39.780817 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.780714    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db2ed92c-6453-4cc6-a686-960dd789a40f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db2ed92c-6453-4cc6-a686-960dd789a40f" (UID: "db2ed92c-6453-4cc6-a686-960dd789a40f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:10:39.879477 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.879438    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db2ed92c-6453-4cc6-a686-960dd789a40f-dshm\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:10:39.879477 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.879469    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqb7r\" (UniqueName: \"kubernetes.io/projected/db2ed92c-6453-4cc6-a686-960dd789a40f-kube-api-access-fqb7r\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:10:39.879477 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.879486    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db2ed92c-6453-4cc6-a686-960dd789a40f-tls-certs\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:10:39.944655 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.944615    2580 generic.go:356] "Generic (PLEG): container finished" podID="db2ed92c-6453-4cc6-a686-960dd789a40f" containerID="97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea" exitCode=0
Jul 21 17:10:39.944655 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.944656    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"
Jul 21 17:10:39.944916 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.944710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh" event={"ID":"db2ed92c-6453-4cc6-a686-960dd789a40f","Type":"ContainerDied","Data":"97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea"}
Jul 21 17:10:39.944916 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.944752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh" event={"ID":"db2ed92c-6453-4cc6-a686-960dd789a40f","Type":"ContainerDied","Data":"86fe57bf646e651fab420413b002a21690b8ecd2c530eabc09b0ef3eb1cb7946"}
Jul 21 17:10:39.944916 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.944774    2580 scope.go:117] "RemoveContainer" containerID="97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea"
Jul 21 17:10:39.954354 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.954333    2580 scope.go:117] "RemoveContainer" containerID="97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea"
Jul 21 17:10:39.954844 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:10:39.954822    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea\": container with ID starting with 97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea not found: ID does not exist" containerID="97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea"
Jul 21 17:10:39.954947 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.954853    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea"} err="failed to get container status \"97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea\": rpc error: code = NotFound desc = could not find container \"97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea\": container with ID starting with 97f350e3ab70e597baf36c14e739285c7e524ddf6254c0774047ad90c7af65ea not found: ID does not exist"
Jul 21 17:10:39.967788 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.967763    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"]
Jul 21 17:10:39.971637 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:39.971610    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/auth-test-model-5b0b89-kserve-f64874475-qbzdh"]
Jul 21 17:10:40.940095 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:10:40.940059    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db2ed92c-6453-4cc6-a686-960dd789a40f" path="/var/lib/kubelet/pods/db2ed92c-6453-4cc6-a686-960dd789a40f/volumes"
Jul 21 17:11:17.098126 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.098094    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"]
Jul 21 17:11:17.098661 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.098425    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db2ed92c-6453-4cc6-a686-960dd789a40f" containerName="main"
Jul 21 17:11:17.098661 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.098438    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db2ed92c-6453-4cc6-a686-960dd789a40f" containerName="main"
Jul 21 17:11:17.098661 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.098492    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db2ed92c-6453-4cc6-a686-960dd789a40f" containerName="main"
Jul 21 17:11:17.101607 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.101585    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.105947 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.105926    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e17cb5\"/\"sub-test-model-e17cb5-kserve-self-signed-certs\""
Jul 21 17:11:17.106054 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.105979    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e17cb5\"/\"openshift-service-ca.crt\""
Jul 21 17:11:17.106304 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.106227    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e17cb5\"/\"default-dockercfg-jhbxf\""
Jul 21 17:11:17.106437 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.106320    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e17cb5\"/\"kube-root-ca.crt\""
Jul 21 17:11:17.123275 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.123252    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"]
Jul 21 17:11:17.191684 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.191652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmdt7\" (UniqueName: \"kubernetes.io/projected/1592ed3b-3518-4019-940d-08a0011b5699-kube-api-access-rmdt7\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.191851 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.191706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-home\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.191851 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.191733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1592ed3b-3518-4019-940d-08a0011b5699-tls-certs\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.191851 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.191761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-model-cache\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.191851 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.191793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-tmp-dir\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.191851 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.191832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-dshm\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.292863 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.292833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-tmp-dir\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.293060 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.292883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-dshm\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.293060 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.292931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmdt7\" (UniqueName: \"kubernetes.io/projected/1592ed3b-3518-4019-940d-08a0011b5699-kube-api-access-rmdt7\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.293060 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.292988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-home\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.293060 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.293014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1592ed3b-3518-4019-940d-08a0011b5699-tls-certs\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.293299 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.293060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-model-cache\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.293299 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.293280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-tmp-dir\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.293413 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.293381    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-model-cache\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.293413 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.293393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-home\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.295399 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.295376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-dshm\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.295614 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.295596    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1592ed3b-3518-4019-940d-08a0011b5699-tls-certs\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.306974 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.306952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmdt7\" (UniqueName: \"kubernetes.io/projected/1592ed3b-3518-4019-940d-08a0011b5699-kube-api-access-rmdt7\") pod \"sub-test-model-e17cb5-kserve-84f98cfd69-w9s47\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") " pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.411939 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.411912    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:17.549090 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:17.549052    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"]
Jul 21 17:11:17.552736 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:11:17.552696    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1592ed3b_3518_4019_940d_08a0011b5699.slice/crio-d7bf04c4fa2a7232da98d430c3694b98e9e3d7faf544fffada796eef1e660c5f WatchSource:0}: Error finding container d7bf04c4fa2a7232da98d430c3694b98e9e3d7faf544fffada796eef1e660c5f: Status 404 returned error can't find the container with id d7bf04c4fa2a7232da98d430c3694b98e9e3d7faf544fffada796eef1e660c5f
Jul 21 17:11:18.117739 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:18.117707    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47" event={"ID":"1592ed3b-3518-4019-940d-08a0011b5699","Type":"ContainerStarted","Data":"0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee"}
Jul 21 17:11:18.117739 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:18.117743    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47" event={"ID":"1592ed3b-3518-4019-940d-08a0011b5699","Type":"ContainerStarted","Data":"d7bf04c4fa2a7232da98d430c3694b98e9e3d7faf544fffada796eef1e660c5f"}
Jul 21 17:11:18.136197 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:18.136152    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47" podStartSLOduration=1.136138911 podStartE2EDuration="1.136138911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:11:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:11:18.13502977 +0000 UTC m=+2573.634391015" watchObservedRunningTime="2026-07-21 17:11:18.136138911 +0000 UTC m=+2573.635500154"
Jul 21 17:11:27.412414 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:27.412376    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:27.412825 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:27.412431    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:27.425303 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:27.425283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:11:28.170823 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:11:28.170794    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:12:06.450546 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:06.450495    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"]
Jul 21 17:12:17.867745 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.867714    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"]
Jul 21 17:12:17.870993 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.870975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.873888 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.873863    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5b0b89\"/\"kube-root-ca.crt\""
Jul 21 17:12:17.874832 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.874814    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5b0b89\"/\"openshift-service-ca.crt\""
Jul 21 17:12:17.875062 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.875044    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5b0b89\"/\"rate-test-model-5b0b89-kserve-self-signed-certs\""
Jul 21 17:12:17.875178 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.875113    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5b0b89\"/\"default-dockercfg-q9b56\""
Jul 21 17:12:17.879556 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.879533    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"]
Jul 21 17:12:17.897296 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.897270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-model-cache\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.897408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.897322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffaa7e5e-ec18-491b-812b-254988acd712-tls-certs\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.897408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.897372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-home\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.897408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.897402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-tmp-dir\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.897530 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.897485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-dshm\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.897574 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.897540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp7wc\" (UniqueName: \"kubernetes.io/projected/ffaa7e5e-ec18-491b-812b-254988acd712-kube-api-access-xp7wc\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.998560 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.998508    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-tmp-dir\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.998708 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.998573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-dshm\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.998708 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.998591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp7wc\" (UniqueName: \"kubernetes.io/projected/ffaa7e5e-ec18-491b-812b-254988acd712-kube-api-access-xp7wc\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.998708 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.998664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-model-cache\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.998833 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.998721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffaa7e5e-ec18-491b-812b-254988acd712-tls-certs\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.998833 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.998756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-home\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.998916 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.998888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-tmp-dir\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.999049 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.999030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-home\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:17.999128 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:17.999067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-model-cache\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:18.000779 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:18.000755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-dshm\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:18.001364 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:18.001346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffaa7e5e-ec18-491b-812b-254988acd712-tls-certs\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:18.007784 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:18.007761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp7wc\" (UniqueName: \"kubernetes.io/projected/ffaa7e5e-ec18-491b-812b-254988acd712-kube-api-access-xp7wc\") pod \"rate-test-model-5b0b89-kserve-d7cf6c595-x94p6\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") " pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:18.180832 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:18.180803    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:18.318825 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:18.318798    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"]
Jul 21 17:12:18.320846 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:12:18.320817    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffaa7e5e_ec18_491b_812b_254988acd712.slice/crio-67f67be9d6a798e4b8f1eed2f7c0cf667614f7dd0bd127b1fa63b7a50357627e WatchSource:0}: Error finding container 67f67be9d6a798e4b8f1eed2f7c0cf667614f7dd0bd127b1fa63b7a50357627e: Status 404 returned error can't find the container with id 67f67be9d6a798e4b8f1eed2f7c0cf667614f7dd0bd127b1fa63b7a50357627e
Jul 21 17:12:18.382151 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:18.382119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6" event={"ID":"ffaa7e5e-ec18-491b-812b-254988acd712","Type":"ContainerStarted","Data":"67f67be9d6a798e4b8f1eed2f7c0cf667614f7dd0bd127b1fa63b7a50357627e"}
Jul 21 17:12:19.388699 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:19.388666    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6" event={"ID":"ffaa7e5e-ec18-491b-812b-254988acd712","Type":"ContainerStarted","Data":"3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5"}
Jul 21 17:12:19.409612 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:19.409564    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6" podStartSLOduration=2.409548592 podStartE2EDuration="2.409548592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:12:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:12:19.409062651 +0000 UTC m=+2634.908423896" watchObservedRunningTime="2026-07-21 17:12:19.409548592 +0000 UTC m=+2634.908909835"
Jul 21 17:12:21.470817 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.470751    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47" podUID="1592ed3b-3518-4019-940d-08a0011b5699" containerName="main" containerID="cri-o://0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee" gracePeriod=45
Jul 21 17:12:21.723914 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.723855    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:12:21.837386 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837356    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-model-cache\") pod \"1592ed3b-3518-4019-940d-08a0011b5699\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") "
Jul 21 17:12:21.837386 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837392    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-home\") pod \"1592ed3b-3518-4019-940d-08a0011b5699\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") "
Jul 21 17:12:21.837677 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837427    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1592ed3b-3518-4019-940d-08a0011b5699-tls-certs\") pod \"1592ed3b-3518-4019-940d-08a0011b5699\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") "
Jul 21 17:12:21.837677 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837453    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmdt7\" (UniqueName: \"kubernetes.io/projected/1592ed3b-3518-4019-940d-08a0011b5699-kube-api-access-rmdt7\") pod \"1592ed3b-3518-4019-940d-08a0011b5699\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") "
Jul 21 17:12:21.837677 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837479    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-tmp-dir\") pod \"1592ed3b-3518-4019-940d-08a0011b5699\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") "
Jul 21 17:12:21.837677 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837502    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-dshm\") pod \"1592ed3b-3518-4019-940d-08a0011b5699\" (UID: \"1592ed3b-3518-4019-940d-08a0011b5699\") "
Jul 21 17:12:21.837905 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837676    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-home" (OuterVolumeSpecName: "home") pod "1592ed3b-3518-4019-940d-08a0011b5699" (UID: "1592ed3b-3518-4019-940d-08a0011b5699"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:12:21.837905 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837689    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-model-cache" (OuterVolumeSpecName: "model-cache") pod "1592ed3b-3518-4019-940d-08a0011b5699" (UID: "1592ed3b-3518-4019-940d-08a0011b5699"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:12:21.837905 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837788    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1592ed3b-3518-4019-940d-08a0011b5699" (UID: "1592ed3b-3518-4019-940d-08a0011b5699"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:12:21.837905 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837843    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-model-cache\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:12:21.837905 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837856    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-home\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:12:21.837905 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.837869    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-tmp-dir\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:12:21.839843 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.839820    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1592ed3b-3518-4019-940d-08a0011b5699-kube-api-access-rmdt7" (OuterVolumeSpecName: "kube-api-access-rmdt7") pod "1592ed3b-3518-4019-940d-08a0011b5699" (UID: "1592ed3b-3518-4019-940d-08a0011b5699"). InnerVolumeSpecName "kube-api-access-rmdt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:12:21.839843 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.839830    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-dshm" (OuterVolumeSpecName: "dshm") pod "1592ed3b-3518-4019-940d-08a0011b5699" (UID: "1592ed3b-3518-4019-940d-08a0011b5699"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:12:21.839843 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.839831    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1592ed3b-3518-4019-940d-08a0011b5699-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1592ed3b-3518-4019-940d-08a0011b5699" (UID: "1592ed3b-3518-4019-940d-08a0011b5699"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:12:21.938417 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.938365    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1592ed3b-3518-4019-940d-08a0011b5699-tls-certs\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:12:21.938417 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.938409    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmdt7\" (UniqueName: \"kubernetes.io/projected/1592ed3b-3518-4019-940d-08a0011b5699-kube-api-access-rmdt7\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:12:21.938417 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:21.938419    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1592ed3b-3518-4019-940d-08a0011b5699-dshm\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:12:22.402962 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:22.402929    2580 generic.go:356] "Generic (PLEG): container finished" podID="1592ed3b-3518-4019-940d-08a0011b5699" containerID="0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee" exitCode=0
Jul 21 17:12:22.403136 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:22.402974    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"
Jul 21 17:12:22.403136 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:22.403017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47" event={"ID":"1592ed3b-3518-4019-940d-08a0011b5699","Type":"ContainerDied","Data":"0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee"}
Jul 21 17:12:22.403136 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:22.403060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47" event={"ID":"1592ed3b-3518-4019-940d-08a0011b5699","Type":"ContainerDied","Data":"d7bf04c4fa2a7232da98d430c3694b98e9e3d7faf544fffada796eef1e660c5f"}
Jul 21 17:12:22.403136 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:22.403079    2580 scope.go:117] "RemoveContainer" containerID="0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee"
Jul 21 17:12:22.411948 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:22.411933    2580 scope.go:117] "RemoveContainer" containerID="0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee"
Jul 21 17:12:22.412223 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:12:22.412205    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee\": container with ID starting with 0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee not found: ID does not exist" containerID="0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee"
Jul 21 17:12:22.412283 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:22.412233    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee"} err="failed to get container status \"0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee\": rpc error: code = NotFound desc = could not find container \"0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee\": container with ID starting with 0bcb7cafc7d4870be1d080c7ed190eb2f76ca72f839c5b9208401fab564755ee not found: ID does not exist"
Jul 21 17:12:22.424938 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:22.424911    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"]
Jul 21 17:12:22.431756 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:22.431735    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e17cb5/sub-test-model-e17cb5-kserve-84f98cfd69-w9s47"]
Jul 21 17:12:22.939743 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:22.939711    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1592ed3b-3518-4019-940d-08a0011b5699" path="/var/lib/kubelet/pods/1592ed3b-3518-4019-940d-08a0011b5699/volumes"
Jul 21 17:12:25.211653 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:25.211620    2580 scope.go:117] "RemoveContainer" containerID="f0c8f0e16752fd46a581438751eceecce285d96928e029c3697c73b950d8c90e"
Jul 21 17:12:25.219864 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:25.219846    2580 scope.go:117] "RemoveContainer" containerID="6f5f3ae29c83f9b59495918cab921005805fbff97ac62da2dba86e3b9d7daf68"
Jul 21 17:12:28.181692 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:28.181646    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:28.181692 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:28.181705    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:28.194333 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:28.194309    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:12:28.443041 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:12:28.442962    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:13:25.246257 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:13:25.246181    2580 scope.go:117] "RemoveContainer" containerID="bb8e24c2a0401a71e231d23ae0024cd1571763a4dc601a49598476d7fe8cb8fb"
Jul 21 17:13:46.008387 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:13:46.008352    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"]
Jul 21 17:14:01.028908 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.028858    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6" podUID="ffaa7e5e-ec18-491b-812b-254988acd712" containerName="main" containerID="cri-o://3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5" gracePeriod=45
Jul 21 17:14:01.276076 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.276053    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:14:01.397400 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-model-cache\") pod \"ffaa7e5e-ec18-491b-812b-254988acd712\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") "
Jul 21 17:14:01.397583 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397416    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffaa7e5e-ec18-491b-812b-254988acd712-tls-certs\") pod \"ffaa7e5e-ec18-491b-812b-254988acd712\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") "
Jul 21 17:14:01.397583 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397432    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-tmp-dir\") pod \"ffaa7e5e-ec18-491b-812b-254988acd712\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") "
Jul 21 17:14:01.397583 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397566    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-home\") pod \"ffaa7e5e-ec18-491b-812b-254988acd712\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") "
Jul 21 17:14:01.397756 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397597    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-dshm\") pod \"ffaa7e5e-ec18-491b-812b-254988acd712\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") "
Jul 21 17:14:01.397756 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397624    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp7wc\" (UniqueName: \"kubernetes.io/projected/ffaa7e5e-ec18-491b-812b-254988acd712-kube-api-access-xp7wc\") pod \"ffaa7e5e-ec18-491b-812b-254988acd712\" (UID: \"ffaa7e5e-ec18-491b-812b-254988acd712\") "
Jul 21 17:14:01.397756 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397639    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-model-cache" (OuterVolumeSpecName: "model-cache") pod "ffaa7e5e-ec18-491b-812b-254988acd712" (UID: "ffaa7e5e-ec18-491b-812b-254988acd712"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:14:01.397756 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397714    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ffaa7e5e-ec18-491b-812b-254988acd712" (UID: "ffaa7e5e-ec18-491b-812b-254988acd712"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:14:01.397965 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397831    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-home" (OuterVolumeSpecName: "home") pod "ffaa7e5e-ec18-491b-812b-254988acd712" (UID: "ffaa7e5e-ec18-491b-812b-254988acd712"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:14:01.397965 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397938    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-tmp-dir\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:01.397965 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397951    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-home\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:01.397965 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.397959    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-model-cache\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:01.399723 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.399697    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffaa7e5e-ec18-491b-812b-254988acd712-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ffaa7e5e-ec18-491b-812b-254988acd712" (UID: "ffaa7e5e-ec18-491b-812b-254988acd712"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:14:01.399838 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.399816    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-dshm" (OuterVolumeSpecName: "dshm") pod "ffaa7e5e-ec18-491b-812b-254988acd712" (UID: "ffaa7e5e-ec18-491b-812b-254988acd712"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:14:01.399883 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.399830    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffaa7e5e-ec18-491b-812b-254988acd712-kube-api-access-xp7wc" (OuterVolumeSpecName: "kube-api-access-xp7wc") pod "ffaa7e5e-ec18-491b-812b-254988acd712" (UID: "ffaa7e5e-ec18-491b-812b-254988acd712"). InnerVolumeSpecName "kube-api-access-xp7wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:14:01.498564 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.498535    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffaa7e5e-ec18-491b-812b-254988acd712-tls-certs\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:01.498564 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.498557    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffaa7e5e-ec18-491b-812b-254988acd712-dshm\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:01.498564 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.498567    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp7wc\" (UniqueName: \"kubernetes.io/projected/ffaa7e5e-ec18-491b-812b-254988acd712-kube-api-access-xp7wc\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:01.842135 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.842105    2580 generic.go:356] "Generic (PLEG): container finished" podID="ffaa7e5e-ec18-491b-812b-254988acd712" containerID="3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5" exitCode=0
Jul 21 17:14:01.842309 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.842146    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"
Jul 21 17:14:01.842309 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.842187    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6" event={"ID":"ffaa7e5e-ec18-491b-812b-254988acd712","Type":"ContainerDied","Data":"3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5"}
Jul 21 17:14:01.842309 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.842226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6" event={"ID":"ffaa7e5e-ec18-491b-812b-254988acd712","Type":"ContainerDied","Data":"67f67be9d6a798e4b8f1eed2f7c0cf667614f7dd0bd127b1fa63b7a50357627e"}
Jul 21 17:14:01.842309 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.842243    2580 scope.go:117] "RemoveContainer" containerID="3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5"
Jul 21 17:14:01.851383 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.851368    2580 scope.go:117] "RemoveContainer" containerID="3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5"
Jul 21 17:14:01.851666 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:14:01.851649    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5\": container with ID starting with 3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5 not found: ID does not exist" containerID="3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5"
Jul 21 17:14:01.851730 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.851675    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5"} err="failed to get container status \"3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5\": rpc error: code = NotFound desc = could not find container \"3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5\": container with ID starting with 3990842c650e056c61143fbbb13279a598c77d3999c4bfe18b901e6f142f1ff5 not found: ID does not exist"
Jul 21 17:14:01.863439 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.863415    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"]
Jul 21 17:14:01.867922 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:01.867904    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5b0b89/rate-test-model-5b0b89-kserve-d7cf6c595-x94p6"]
Jul 21 17:14:02.939084 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:02.939050    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffaa7e5e-ec18-491b-812b-254988acd712" path="/var/lib/kubelet/pods/ffaa7e5e-ec18-491b-812b-254988acd712/volumes"
Jul 21 17:14:03.360982 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.360903    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7497-139965123565584"]
Jul 21 17:14:03.361210 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.361198    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1592ed3b-3518-4019-940d-08a0011b5699" containerName="main"
Jul 21 17:14:03.361267 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.361211    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1592ed3b-3518-4019-940d-08a0011b5699" containerName="main"
Jul 21 17:14:03.361267 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.361222    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffaa7e5e-ec18-491b-812b-254988acd712" containerName="main"
Jul 21 17:14:03.361267 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.361227    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffaa7e5e-ec18-491b-812b-254988acd712" containerName="main"
Jul 21 17:14:03.361373 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.361288    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffaa7e5e-ec18-491b-812b-254988acd712" containerName="main"
Jul 21 17:14:03.361373 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.361295    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1592ed3b-3518-4019-940d-08a0011b5699" containerName="main"
Jul 21 17:14:03.364355 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.364340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7497-139965123565584"
Jul 21 17:14:03.366887 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.366865    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w8qh4\""
Jul 21 17:14:03.371603 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.371336    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7497-139965123565584"]
Jul 21 17:14:03.412992 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.412969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74pz2\" (UniqueName: \"kubernetes.io/projected/8fc41bb7-90a3-47da-b147-ee07224a10cc-kube-api-access-74pz2\") pod \"test-curl-7497-139965123565584\" (UID: \"8fc41bb7-90a3-47da-b147-ee07224a10cc\") " pod="opendatahub/test-curl-7497-139965123565584"
Jul 21 17:14:03.514241 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.514214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74pz2\" (UniqueName: \"kubernetes.io/projected/8fc41bb7-90a3-47da-b147-ee07224a10cc-kube-api-access-74pz2\") pod \"test-curl-7497-139965123565584\" (UID: \"8fc41bb7-90a3-47da-b147-ee07224a10cc\") " pod="opendatahub/test-curl-7497-139965123565584"
Jul 21 17:14:03.523835 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.523804    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74pz2\" (UniqueName: \"kubernetes.io/projected/8fc41bb7-90a3-47da-b147-ee07224a10cc-kube-api-access-74pz2\") pod \"test-curl-7497-139965123565584\" (UID: \"8fc41bb7-90a3-47da-b147-ee07224a10cc\") " pod="opendatahub/test-curl-7497-139965123565584"
Jul 21 17:14:03.673896 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.673855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7497-139965123565584"
Jul 21 17:14:03.803388 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.803363    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7497-139965123565584"]
Jul 21 17:14:03.805411 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:14:03.805385    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fc41bb7_90a3_47da_b147_ee07224a10cc.slice/crio-3a3ab2b1e4a2893a0464ebdaf60f4c024766a21c82a6e0ba5aab771beeac86d8 WatchSource:0}: Error finding container 3a3ab2b1e4a2893a0464ebdaf60f4c024766a21c82a6e0ba5aab771beeac86d8: Status 404 returned error can't find the container with id 3a3ab2b1e4a2893a0464ebdaf60f4c024766a21c82a6e0ba5aab771beeac86d8
Jul 21 17:14:03.806643 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.806624    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 17:14:03.853489 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:03.853464    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7497-139965123565584" event={"ID":"8fc41bb7-90a3-47da-b147-ee07224a10cc","Type":"ContainerStarted","Data":"3a3ab2b1e4a2893a0464ebdaf60f4c024766a21c82a6e0ba5aab771beeac86d8"}
Jul 21 17:14:04.858977 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:04.858886    2580 generic.go:356] "Generic (PLEG): container finished" podID="8fc41bb7-90a3-47da-b147-ee07224a10cc" containerID="6d380ced3d6c505116a4782d80b51190fdddb73466f0838b83b9e1081ab431d5" exitCode=0
Jul 21 17:14:04.859330 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:04.858971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7497-139965123565584" event={"ID":"8fc41bb7-90a3-47da-b147-ee07224a10cc","Type":"ContainerDied","Data":"6d380ced3d6c505116a4782d80b51190fdddb73466f0838b83b9e1081ab431d5"}
Jul 21 17:14:05.989435 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:05.989414    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7497-139965123565584"
Jul 21 17:14:06.026879 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.026854    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7497-139965123565584_8fc41bb7-90a3-47da-b147-ee07224a10cc/test-curl-7497-139965123565584/0.log"
Jul 21 17:14:06.034002 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.033983    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74pz2\" (UniqueName: \"kubernetes.io/projected/8fc41bb7-90a3-47da-b147-ee07224a10cc-kube-api-access-74pz2\") pod \"8fc41bb7-90a3-47da-b147-ee07224a10cc\" (UID: \"8fc41bb7-90a3-47da-b147-ee07224a10cc\") "
Jul 21 17:14:06.036200 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.036175    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fc41bb7-90a3-47da-b147-ee07224a10cc-kube-api-access-74pz2" (OuterVolumeSpecName: "kube-api-access-74pz2") pod "8fc41bb7-90a3-47da-b147-ee07224a10cc" (UID: "8fc41bb7-90a3-47da-b147-ee07224a10cc"). InnerVolumeSpecName "kube-api-access-74pz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:14:06.049947 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.049925    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7497-139965123565584"]
Jul 21 17:14:06.053734 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.053715    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7497-139965123565584"]
Jul 21 17:14:06.135431 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.135376    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74pz2\" (UniqueName: \"kubernetes.io/projected/8fc41bb7-90a3-47da-b147-ee07224a10cc-kube-api-access-74pz2\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:06.237102 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.237081    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"]
Jul 21 17:14:06.237388 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.237377    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fc41bb7-90a3-47da-b147-ee07224a10cc" containerName="test-curl-7497-139965123565584"
Jul 21 17:14:06.237438 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.237391    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fc41bb7-90a3-47da-b147-ee07224a10cc" containerName="test-curl-7497-139965123565584"
Jul 21 17:14:06.237474 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.237459    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fc41bb7-90a3-47da-b147-ee07224a10cc" containerName="test-curl-7497-139965123565584"
Jul 21 17:14:06.240317 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.240302    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"
Jul 21 17:14:06.242973 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.242953    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 17:14:06.247742 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.247722    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"]
Jul 21 17:14:06.337052 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.337020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv\" (UID: \"7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"
Jul 21 17:14:06.438458 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.438433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv\" (UID: \"7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"
Jul 21 17:14:06.439008 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.438990    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv\" (UID: \"7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"
Jul 21 17:14:06.551228 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.551203    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"
Jul 21 17:14:06.682080 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.682055    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"]
Jul 21 17:14:06.684162 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:14:06.684134    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d08c6c7_4b9c_4b8b_8b3a_df62b3487cb4.slice/crio-cadcdf77548e01c1a7cc25f75b25a1a81f9bf4c8c16866b0b47ccfbf864200b8 WatchSource:0}: Error finding container cadcdf77548e01c1a7cc25f75b25a1a81f9bf4c8c16866b0b47ccfbf864200b8: Status 404 returned error can't find the container with id cadcdf77548e01c1a7cc25f75b25a1a81f9bf4c8c16866b0b47ccfbf864200b8
Jul 21 17:14:06.868950 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.868923    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv" event={"ID":"7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4","Type":"ContainerStarted","Data":"cadcdf77548e01c1a7cc25f75b25a1a81f9bf4c8c16866b0b47ccfbf864200b8"}
Jul 21 17:14:06.870218 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.870198    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a3ab2b1e4a2893a0464ebdaf60f4c024766a21c82a6e0ba5aab771beeac86d8"
Jul 21 17:14:06.870330 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.870221    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7497-139965123565584"
Jul 21 17:14:06.879767 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:14:06.879738    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7497-139965123565584\" is forbidden: User \"system:node:ip-10-0-141-158.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-158.ec2.internal' and this object" podUID="8fc41bb7-90a3-47da-b147-ee07224a10cc" pod="opendatahub/test-curl-7497-139965123565584"
Jul 21 17:14:06.938820 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:06.938792    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fc41bb7-90a3-47da-b147-ee07224a10cc" path="/var/lib/kubelet/pods/8fc41bb7-90a3-47da-b147-ee07224a10cc/volumes"
Jul 21 17:14:07.875453 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:07.875418    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4" containerID="e2f6d90fd1cfd620c90baa2eee32b123f7ecfbaa511ede44998f8e56ea9d3169" exitCode=0
Jul 21 17:14:07.875892 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:07.875476    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv" event={"ID":"7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4","Type":"ContainerDied","Data":"e2f6d90fd1cfd620c90baa2eee32b123f7ecfbaa511ede44998f8e56ea9d3169"}
Jul 21 17:14:09.006881 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:09.006859    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"
Jul 21 17:14:09.060531 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:09.060481    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4-maas-api-service-ca\") pod \"7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4\" (UID: \"7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4\") "
Jul 21 17:14:09.060859 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:09.060839    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4" (UID: "7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:14:09.161877 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:09.161857    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:09.884866 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:09.884834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv" event={"ID":"7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4","Type":"ContainerDied","Data":"cadcdf77548e01c1a7cc25f75b25a1a81f9bf4c8c16866b0b47ccfbf864200b8"}
Jul 21 17:14:09.884866 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:09.884849    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"
Jul 21 17:14:09.885052 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:09.884879    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cadcdf77548e01c1a7cc25f75b25a1a81f9bf4c8c16866b0b47ccfbf864200b8"
Jul 21 17:14:16.841758 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:16.841727    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"]
Jul 21 17:14:16.842260 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:16.842007    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9" podUID="c0e36ac7-7518-4cd7-bfb3-2d9f63598436" containerName="maas-api" containerID="cri-o://461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb" gracePeriod=30
Jul 21 17:14:17.082619 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.082596    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:14:17.225739 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.225710    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-maas-api-tls\") pod \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") "
Jul 21 17:14:17.225912 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.225755    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kff4\" (UniqueName: \"kubernetes.io/projected/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-kube-api-access-6kff4\") pod \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") "
Jul 21 17:14:17.225912 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.225785    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-openshift-service-ca\") pod \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\" (UID: \"c0e36ac7-7518-4cd7-bfb3-2d9f63598436\") "
Jul 21 17:14:17.226210 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.226180    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0e36ac7-7518-4cd7-bfb3-2d9f63598436" (UID: "c0e36ac7-7518-4cd7-bfb3-2d9f63598436"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:14:17.228073 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.228044    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0e36ac7-7518-4cd7-bfb3-2d9f63598436" (UID: "c0e36ac7-7518-4cd7-bfb3-2d9f63598436"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:14:17.228073 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.228065    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-kube-api-access-6kff4" (OuterVolumeSpecName: "kube-api-access-6kff4") pod "c0e36ac7-7518-4cd7-bfb3-2d9f63598436" (UID: "c0e36ac7-7518-4cd7-bfb3-2d9f63598436"). InnerVolumeSpecName "kube-api-access-6kff4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:14:17.326761 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.326736    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:17.326761 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.326759    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kff4\" (UniqueName: \"kubernetes.io/projected/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-kube-api-access-6kff4\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:17.326901 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.326769    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0e36ac7-7518-4cd7-bfb3-2d9f63598436-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:17.925395 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.925363    2580 generic.go:356] "Generic (PLEG): container finished" podID="c0e36ac7-7518-4cd7-bfb3-2d9f63598436" containerID="461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb" exitCode=0
Jul 21 17:14:17.925850 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.925410    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"
Jul 21 17:14:17.925850 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.925449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9" event={"ID":"c0e36ac7-7518-4cd7-bfb3-2d9f63598436","Type":"ContainerDied","Data":"461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb"}
Jul 21 17:14:17.925850 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.925506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9" event={"ID":"c0e36ac7-7518-4cd7-bfb3-2d9f63598436","Type":"ContainerDied","Data":"a03c6dd03eaa7ade7409c86094cc910fabf29faa2d3f26ae25e57894c1208849"}
Jul 21 17:14:17.925850 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.925540    2580 scope.go:117] "RemoveContainer" containerID="461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb"
Jul 21 17:14:17.934737 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.934709    2580 scope.go:117] "RemoveContainer" containerID="461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb"
Jul 21 17:14:17.934973 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:14:17.934955    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb\": container with ID starting with 461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb not found: ID does not exist" containerID="461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb"
Jul 21 17:14:17.935040 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.934980    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb"} err="failed to get container status \"461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb\": rpc error: code = NotFound desc = could not find container \"461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb\": container with ID starting with 461232a7a20d92fa0729e6c05555c70f8c04456a2715058542a3ddee8d6e96cb not found: ID does not exist"
Jul 21 17:14:17.947310 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.947285    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"]
Jul 21 17:14:17.949534 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:17.949500    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b0b89-5f44f8d6cd-ns6c9"]
Jul 21 17:14:18.939045 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:18.939009    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0e36ac7-7518-4cd7-bfb3-2d9f63598436" path="/var/lib/kubelet/pods/c0e36ac7-7518-4cd7-bfb3-2d9f63598436/volumes"
Jul 21 17:14:21.344750 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:21.344720    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"]
Jul 21 17:14:21.348097 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:21.348074    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b0b89-r48sv"]
Jul 21 17:14:22.939642 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:22.939611    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4" path="/var/lib/kubelet/pods/7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4/volumes"
Jul 21 17:14:28.489449 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.489412    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"]
Jul 21 17:14:28.489859 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.489726    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4" containerName="revoke-keys"
Jul 21 17:14:28.489859 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.489738    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4" containerName="revoke-keys"
Jul 21 17:14:28.489859 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.489755    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0e36ac7-7518-4cd7-bfb3-2d9f63598436" containerName="maas-api"
Jul 21 17:14:28.489859 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.489761    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0e36ac7-7518-4cd7-bfb3-2d9f63598436" containerName="maas-api"
Jul 21 17:14:28.489859 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.489821    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0e36ac7-7518-4cd7-bfb3-2d9f63598436" containerName="maas-api"
Jul 21 17:14:28.489859 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.489830    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d08c6c7-4b9c-4b8b-8b3a-df62b3487cb4" containerName="revoke-keys"
Jul 21 17:14:28.494043 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.494026    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"
Jul 21 17:14:28.496474 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.496451    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 17:14:28.500247 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.500163    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"]
Jul 21 17:14:28.618457 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.618421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f91010-b93a-4465-b146-6ab4defff6bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv\" (UID: \"c5f91010-b93a-4465-b146-6ab4defff6bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"
Jul 21 17:14:28.719097 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.719065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f91010-b93a-4465-b146-6ab4defff6bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv\" (UID: \"c5f91010-b93a-4465-b146-6ab4defff6bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"
Jul 21 17:14:28.719678 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.719661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f91010-b93a-4465-b146-6ab4defff6bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv\" (UID: \"c5f91010-b93a-4465-b146-6ab4defff6bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"
Jul 21 17:14:28.806050 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.805986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"
Jul 21 17:14:28.945840 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.945817    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"]
Jul 21 17:14:28.947918 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:14:28.947894    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5f91010_b93a_4465_b146_6ab4defff6bd.slice/crio-4df7856a1e1e68ca99b0fc426c1e928426110f2d9a139f8e3a8c7ae2bac6112a WatchSource:0}: Error finding container 4df7856a1e1e68ca99b0fc426c1e928426110f2d9a139f8e3a8c7ae2bac6112a: Status 404 returned error can't find the container with id 4df7856a1e1e68ca99b0fc426c1e928426110f2d9a139f8e3a8c7ae2bac6112a
Jul 21 17:14:28.975426 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:28.975396    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv" event={"ID":"c5f91010-b93a-4465-b146-6ab4defff6bd","Type":"ContainerStarted","Data":"4df7856a1e1e68ca99b0fc426c1e928426110f2d9a139f8e3a8c7ae2bac6112a"}
Jul 21 17:14:29.981353 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:29.981317    2580 generic.go:356] "Generic (PLEG): container finished" podID="c5f91010-b93a-4465-b146-6ab4defff6bd" containerID="ee4f4449e6f48d7e53225da8c8927619bf32b58e5981eedadcb7a57512c76a0a" exitCode=0
Jul 21 17:14:29.981745 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:29.981378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv" event={"ID":"c5f91010-b93a-4465-b146-6ab4defff6bd","Type":"ContainerDied","Data":"ee4f4449e6f48d7e53225da8c8927619bf32b58e5981eedadcb7a57512c76a0a"}
Jul 21 17:14:31.109286 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:31.109263    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"
Jul 21 17:14:31.240784 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:31.240707    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f91010-b93a-4465-b146-6ab4defff6bd-maas-api-service-ca\") pod \"c5f91010-b93a-4465-b146-6ab4defff6bd\" (UID: \"c5f91010-b93a-4465-b146-6ab4defff6bd\") "
Jul 21 17:14:31.241084 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:31.241062    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5f91010-b93a-4465-b146-6ab4defff6bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5f91010-b93a-4465-b146-6ab4defff6bd" (UID: "c5f91010-b93a-4465-b146-6ab4defff6bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:14:31.341276 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:31.341254    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f91010-b93a-4465-b146-6ab4defff6bd-maas-api-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:31.990482 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:31.990454    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"
Jul 21 17:14:31.990737 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:31.990486    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv" event={"ID":"c5f91010-b93a-4465-b146-6ab4defff6bd","Type":"ContainerDied","Data":"4df7856a1e1e68ca99b0fc426c1e928426110f2d9a139f8e3a8c7ae2bac6112a"}
Jul 21 17:14:31.990737 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:31.990546    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4df7856a1e1e68ca99b0fc426c1e928426110f2d9a139f8e3a8c7ae2bac6112a"
Jul 21 17:14:39.043240 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.043206    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"]
Jul 21 17:14:39.043689 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.043441    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6" podUID="20a31ed9-5e5c-41ad-a605-9de6328f17e5" containerName="maas-api" containerID="cri-o://17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb" gracePeriod=30
Jul 21 17:14:39.294725 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.294668    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:14:39.307192 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.307172    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a31ed9-5e5c-41ad-a605-9de6328f17e5-openshift-service-ca\") pod \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") "
Jul 21 17:14:39.307269 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.307256    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m72md\" (UniqueName: \"kubernetes.io/projected/20a31ed9-5e5c-41ad-a605-9de6328f17e5-kube-api-access-m72md\") pod \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") "
Jul 21 17:14:39.307308 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.307300    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a31ed9-5e5c-41ad-a605-9de6328f17e5-maas-api-tls\") pod \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\" (UID: \"20a31ed9-5e5c-41ad-a605-9de6328f17e5\") "
Jul 21 17:14:39.307531 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.307492    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a31ed9-5e5c-41ad-a605-9de6328f17e5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20a31ed9-5e5c-41ad-a605-9de6328f17e5" (UID: "20a31ed9-5e5c-41ad-a605-9de6328f17e5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:14:39.309319 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.309299    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a31ed9-5e5c-41ad-a605-9de6328f17e5-kube-api-access-m72md" (OuterVolumeSpecName: "kube-api-access-m72md") pod "20a31ed9-5e5c-41ad-a605-9de6328f17e5" (UID: "20a31ed9-5e5c-41ad-a605-9de6328f17e5"). InnerVolumeSpecName "kube-api-access-m72md". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:14:39.309409 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.309382    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20a31ed9-5e5c-41ad-a605-9de6328f17e5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20a31ed9-5e5c-41ad-a605-9de6328f17e5" (UID: "20a31ed9-5e5c-41ad-a605-9de6328f17e5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:14:39.407876 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.407846    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a31ed9-5e5c-41ad-a605-9de6328f17e5-openshift-service-ca\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:39.407876 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.407870    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m72md\" (UniqueName: \"kubernetes.io/projected/20a31ed9-5e5c-41ad-a605-9de6328f17e5-kube-api-access-m72md\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:39.407876 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:39.407883    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a31ed9-5e5c-41ad-a605-9de6328f17e5-maas-api-tls\") on node \"ip-10-0-141-158.ec2.internal\" DevicePath \"\""
Jul 21 17:14:40.023706 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:40.023670    2580 generic.go:356] "Generic (PLEG): container finished" podID="20a31ed9-5e5c-41ad-a605-9de6328f17e5" containerID="17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb" exitCode=0
Jul 21 17:14:40.023906 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:40.023716    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"
Jul 21 17:14:40.023906 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:40.023766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6" event={"ID":"20a31ed9-5e5c-41ad-a605-9de6328f17e5","Type":"ContainerDied","Data":"17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb"}
Jul 21 17:14:40.023906 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:40.023815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6" event={"ID":"20a31ed9-5e5c-41ad-a605-9de6328f17e5","Type":"ContainerDied","Data":"dd88258462c1f4eeffba2027b42122a8ad95f4a8dbd0a37a984d1d96452a039e"}
Jul 21 17:14:40.023906 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:40.023843    2580 scope.go:117] "RemoveContainer" containerID="17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb"
Jul 21 17:14:40.032037 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:40.032021    2580 scope.go:117] "RemoveContainer" containerID="17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb"
Jul 21 17:14:40.032280 ip-10-0-141-158 kubenswrapper[2580]: E0721 17:14:40.032264    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb\": container with ID starting with 17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb not found: ID does not exist" containerID="17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb"
Jul 21 17:14:40.032329 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:40.032289    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb"} err="failed to get container status \"17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb\": rpc error: code = NotFound desc = could not find container \"17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb\": container with ID starting with 17a65a2a5fac4f4c228761bcb63132119e9ece62b2a878c5b336d0da398078bb not found: ID does not exist"
Jul 21 17:14:40.046464 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:40.046445    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"]
Jul 21 17:14:40.050492 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:40.050470    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e17cb5-5c5fc4895d-jnlt6"]
Jul 21 17:14:40.939856 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:40.939822    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20a31ed9-5e5c-41ad-a605-9de6328f17e5" path="/var/lib/kubelet/pods/20a31ed9-5e5c-41ad-a605-9de6328f17e5/volumes"
Jul 21 17:14:43.574686 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:43.574649    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"]
Jul 21 17:14:43.581998 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:43.581975    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e17cb5-48gwv"]
Jul 21 17:14:44.942597 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:44.942554    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5f91010-b93a-4465-b146-6ab4defff6bd" path="/var/lib/kubelet/pods/c5f91010-b93a-4465-b146-6ab4defff6bd/volumes"
Jul 21 17:14:57.509668 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:57.509636    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87c9445c4-n6cdx_aec37a82-46f6-46a3-8c24-37985db8b9d1/copy-manifests/0.log"
Jul 21 17:14:57.516407 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:57.516386    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87c9445c4-n6cdx_aec37a82-46f6-46a3-8c24-37985db8b9d1/manager/0.log"
Jul 21 17:14:57.957482 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:57.957446    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744205-qmhj9_3a5d5552-19e6-486e-94bf-c6a91b39b987/cleanup/0.log"
Jul 21 17:14:58.065942 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:14:58.065910    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744220-kn54r_55dc8451-4a20-4b99-9712-c88bac8ebfa3/cleanup/0.log"
Jul 21 17:15:00.158610 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.158579    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f"]
Jul 21 17:15:00.158986 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.158895    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5f91010-b93a-4465-b146-6ab4defff6bd" containerName="revoke-keys"
Jul 21 17:15:00.158986 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.158906    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5f91010-b93a-4465-b146-6ab4defff6bd" containerName="revoke-keys"
Jul 21 17:15:00.158986 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.158915    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a31ed9-5e5c-41ad-a605-9de6328f17e5" containerName="maas-api"
Jul 21 17:15:00.158986 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.158920    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a31ed9-5e5c-41ad-a605-9de6328f17e5" containerName="maas-api"
Jul 21 17:15:00.158986 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.158969    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5f91010-b93a-4465-b146-6ab4defff6bd" containerName="revoke-keys"
Jul 21 17:15:00.158986 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.158975    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a31ed9-5e5c-41ad-a605-9de6328f17e5" containerName="maas-api"
Jul 21 17:15:00.162936 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.162920    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f"
Jul 21 17:15:00.170233 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.170213    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5fqhm\""
Jul 21 17:15:00.173544 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.173507    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f"]
Jul 21 17:15:00.173831 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.173816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f"
Jul 21 17:15:00.307272 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.307241    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f"]
Jul 21 17:15:00.310077 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:15:00.310049    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f0e20e_d141_407f_b420_82467ddd6991.slice/crio-cd98175e78330df9697448e8969ba2fa29b060080b36996346f5c041a71c7a65 WatchSource:0}: Error finding container cd98175e78330df9697448e8969ba2fa29b060080b36996346f5c041a71c7a65: Status 404 returned error can't find the container with id cd98175e78330df9697448e8969ba2fa29b060080b36996346f5c041a71c7a65
Jul 21 17:15:00.625168 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.625143    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mk8pp_2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d/discovery/0.log"
Jul 21 17:15:00.836845 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:00.836763    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d77f95f8-tx4tj_cd7875a7-f877-4a29-aa5f-6f3dc41ebb35/kube-auth-proxy/0.log"
Jul 21 17:15:01.110924 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:01.110836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f" event={"ID":"14f0e20e-d141-407f-b420-82467ddd6991","Type":"ContainerStarted","Data":"e06246c848eb9354e5adc8eab0b7a565b2edf269ec7b576f331d0bade14f323e"}
Jul 21 17:15:01.110924 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:01.110895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f" event={"ID":"14f0e20e-d141-407f-b420-82467ddd6991","Type":"ContainerStarted","Data":"cd98175e78330df9697448e8969ba2fa29b060080b36996346f5c041a71c7a65"}
Jul 21 17:15:01.131563 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:01.131499    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f" podStartSLOduration=1.131484158 podStartE2EDuration="1.131484158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:15:01.13009741 +0000 UTC m=+2796.629458659" watchObservedRunningTime="2026-07-21 17:15:01.131484158 +0000 UTC m=+2796.630845402"
Jul 21 17:15:01.877817 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:01.877773    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hb94w_5b65ccf2-bf4c-4f68-b4eb-9c8556901c08/main/0.log"
Jul 21 17:15:02.115954 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:02.115920    2580 generic.go:356] "Generic (PLEG): container finished" podID="14f0e20e-d141-407f-b420-82467ddd6991" containerID="e06246c848eb9354e5adc8eab0b7a565b2edf269ec7b576f331d0bade14f323e" exitCode=0
Jul 21 17:15:02.116089 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:02.115991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f" event={"ID":"14f0e20e-d141-407f-b420-82467ddd6991","Type":"ContainerDied","Data":"e06246c848eb9354e5adc8eab0b7a565b2edf269ec7b576f331d0bade14f323e"}
Jul 21 17:15:03.248043 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:03.248020    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f"
Jul 21 17:15:04.125501 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:04.125471    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f" event={"ID":"14f0e20e-d141-407f-b420-82467ddd6991","Type":"ContainerDied","Data":"cd98175e78330df9697448e8969ba2fa29b060080b36996346f5c041a71c7a65"}
Jul 21 17:15:04.125697 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:04.125485    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744235-ndv5f"
Jul 21 17:15:04.125697 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:04.125541    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd98175e78330df9697448e8969ba2fa29b060080b36996346f5c041a71c7a65"
Jul 21 17:15:07.167397 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.167370    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nbkhg/must-gather-zklk5"]
Jul 21 17:15:07.167765 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.167722    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14f0e20e-d141-407f-b420-82467ddd6991" containerName="cleanup"
Jul 21 17:15:07.167765 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.167734    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="14f0e20e-d141-407f-b420-82467ddd6991" containerName="cleanup"
Jul 21 17:15:07.167840 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.167797    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="14f0e20e-d141-407f-b420-82467ddd6991" containerName="cleanup"
Jul 21 17:15:07.170253 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.170238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nbkhg/must-gather-zklk5"
Jul 21 17:15:07.173998 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.173977    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nbkhg\"/\"kube-root-ca.crt\""
Jul 21 17:15:07.174121 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.174105    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nbkhg\"/\"openshift-service-ca.crt\""
Jul 21 17:15:07.185324 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.185304    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nbkhg/must-gather-zklk5"]
Jul 21 17:15:07.235349 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.235322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e5c75b84-865e-4819-a2a9-392114e7e55d-must-gather-output\") pod \"must-gather-zklk5\" (UID: \"e5c75b84-865e-4819-a2a9-392114e7e55d\") " pod="openshift-must-gather-nbkhg/must-gather-zklk5"
Jul 21 17:15:07.235445 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.235371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qth9p\" (UniqueName: \"kubernetes.io/projected/e5c75b84-865e-4819-a2a9-392114e7e55d-kube-api-access-qth9p\") pod \"must-gather-zklk5\" (UID: \"e5c75b84-865e-4819-a2a9-392114e7e55d\") " pod="openshift-must-gather-nbkhg/must-gather-zklk5"
Jul 21 17:15:07.336669 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.336645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e5c75b84-865e-4819-a2a9-392114e7e55d-must-gather-output\") pod \"must-gather-zklk5\" (UID: \"e5c75b84-865e-4819-a2a9-392114e7e55d\") " pod="openshift-must-gather-nbkhg/must-gather-zklk5"
Jul 21 17:15:07.336768 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.336688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qth9p\" (UniqueName: \"kubernetes.io/projected/e5c75b84-865e-4819-a2a9-392114e7e55d-kube-api-access-qth9p\") pod \"must-gather-zklk5\" (UID: \"e5c75b84-865e-4819-a2a9-392114e7e55d\") " pod="openshift-must-gather-nbkhg/must-gather-zklk5"
Jul 21 17:15:07.336960 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.336941    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e5c75b84-865e-4819-a2a9-392114e7e55d-must-gather-output\") pod \"must-gather-zklk5\" (UID: \"e5c75b84-865e-4819-a2a9-392114e7e55d\") " pod="openshift-must-gather-nbkhg/must-gather-zklk5"
Jul 21 17:15:07.344952 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.344933    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qth9p\" (UniqueName: \"kubernetes.io/projected/e5c75b84-865e-4819-a2a9-392114e7e55d-kube-api-access-qth9p\") pod \"must-gather-zklk5\" (UID: \"e5c75b84-865e-4819-a2a9-392114e7e55d\") " pod="openshift-must-gather-nbkhg/must-gather-zklk5"
Jul 21 17:15:07.479876 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.479821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nbkhg/must-gather-zklk5"
Jul 21 17:15:07.611308 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:07.611165    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nbkhg/must-gather-zklk5"]
Jul 21 17:15:07.613569 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:15:07.613541    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5c75b84_865e_4819_a2a9_392114e7e55d.slice/crio-b04ddf81520fccbadcfff7da76076c284d7409452a24faafaafb0b9711c31e26 WatchSource:0}: Error finding container b04ddf81520fccbadcfff7da76076c284d7409452a24faafaafb0b9711c31e26: Status 404 returned error can't find the container with id b04ddf81520fccbadcfff7da76076c284d7409452a24faafaafb0b9711c31e26
Jul 21 17:15:08.144621 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:08.144588    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbkhg/must-gather-zklk5" event={"ID":"e5c75b84-865e-4819-a2a9-392114e7e55d","Type":"ContainerStarted","Data":"b04ddf81520fccbadcfff7da76076c284d7409452a24faafaafb0b9711c31e26"}
Jul 21 17:15:09.155743 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:09.155220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbkhg/must-gather-zklk5" event={"ID":"e5c75b84-865e-4819-a2a9-392114e7e55d","Type":"ContainerStarted","Data":"f613ae89e228953b329ef6294369d6e6a98edd073317416bff724c5704b0af6c"}
Jul 21 17:15:09.155743 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:09.155279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbkhg/must-gather-zklk5" event={"ID":"e5c75b84-865e-4819-a2a9-392114e7e55d","Type":"ContainerStarted","Data":"c07ef00fa87ca5cb22c0d034b46366f84f1151aae974d8c445271ad4f6473136"}
Jul 21 17:15:09.172294 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:09.172237    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nbkhg/must-gather-zklk5" podStartSLOduration=1.051025035 podStartE2EDuration="2.172218869s" totalImagesPullingTime="1.121193834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:15:09.171028258 +0000 UTC m=+2804.670389502" watchObservedRunningTime="2026-07-21 17:15:09.172218869 +0000 UTC m=+2804.671580113"
Jul 21 17:15:10.421872 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:10.421839    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h4rpx_77f6aedc-bb21-4183-bce6-420144e434b7/global-pull-secret-syncer/0.log"
Jul 21 17:15:10.542012 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:10.541982    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gs24x_e8de554a-353b-4846-bd7f-79b3c8bf0311/konnectivity-agent/0.log"
Jul 21 17:15:10.605778 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:10.605743    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-158.ec2.internal_0609c9a26f287e8753e54b86bb9c45d6/haproxy/0.log"
Jul 21 17:15:18.130572 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.130546    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-d5d9cf589-7hdrl_ddd8a70b-6d1c-465a-a4db-f224964f7416/metrics-server/0.log"
Jul 21 17:15:18.169496 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.169441    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-46hwk_b4b94a8a-ea7d-4b7b-bc45-1bc371c6ef2e/monitoring-plugin/0.log"
Jul 21 17:15:18.281007 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.280898    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f7mf5_d72670a4-5a3a-43b6-9b1f-2ec241b90647/node-exporter/0.log"
Jul 21 17:15:18.315945 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.315914    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f7mf5_d72670a4-5a3a-43b6-9b1f-2ec241b90647/kube-rbac-proxy/0.log"
Jul 21 17:15:18.337878 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.337844    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f7mf5_d72670a4-5a3a-43b6-9b1f-2ec241b90647/init-textfile/0.log"
Jul 21 17:15:18.458771 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.458740    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-bcxcd_bbbaca30-b505-4768-80e6-34c932966aa7/kube-rbac-proxy-main/0.log"
Jul 21 17:15:18.482748 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.482712    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-bcxcd_bbbaca30-b505-4768-80e6-34c932966aa7/kube-rbac-proxy-self/0.log"
Jul 21 17:15:18.505194 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.505141    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-bcxcd_bbbaca30-b505-4768-80e6-34c932966aa7/openshift-state-metrics/0.log"
Jul 21 17:15:18.544501 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.544426    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ea6de6a3-750e-44e9-970f-3ce857090c0d/prometheus/0.log"
Jul 21 17:15:18.567081 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.567054    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ea6de6a3-750e-44e9-970f-3ce857090c0d/config-reloader/0.log"
Jul 21 17:15:18.597355 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.597331    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ea6de6a3-750e-44e9-970f-3ce857090c0d/thanos-sidecar/0.log"
Jul 21 17:15:18.622693 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.622665    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ea6de6a3-750e-44e9-970f-3ce857090c0d/kube-rbac-proxy-web/0.log"
Jul 21 17:15:18.643263 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.643234    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ea6de6a3-750e-44e9-970f-3ce857090c0d/kube-rbac-proxy/0.log"
Jul 21 17:15:18.664381 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.664359    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ea6de6a3-750e-44e9-970f-3ce857090c0d/kube-rbac-proxy-thanos/0.log"
Jul 21 17:15:18.686557 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.686534    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ea6de6a3-750e-44e9-970f-3ce857090c0d/init-config-reloader/0.log"
Jul 21 17:15:18.719552 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.719476    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-pq4j8_bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb/prometheus-operator/0.log"
Jul 21 17:15:18.745247 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.745217    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-pq4j8_bb3ff912-f53c-4bc6-bae5-403ab5a6bcfb/kube-rbac-proxy/0.log"
Jul 21 17:15:18.788064 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:18.788036    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-ppqjx_96c47679-0ac0-4013-bf75-300d948e68ad/prometheus-operator-admission-webhook/0.log"
Jul 21 17:15:20.899612 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:20.899578    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"]
Jul 21 17:15:20.904504 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:20.904469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:20.908526 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:20.908485    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nbkhg\"/\"default-dockercfg-mwnqs\""
Jul 21 17:15:20.915255 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:20.915211    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"]
Jul 21 17:15:20.992635 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:20.992598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-podres\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:20.992635 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:20.992643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfczq\" (UniqueName: \"kubernetes.io/projected/cdc62982-b876-409c-9c0b-a2daa034bb0f-kube-api-access-lfczq\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:20.992906 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:20.992676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-proc\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:20.992906 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:20.992773    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-lib-modules\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:20.992906 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:20.992820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-sys\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.094220 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.094183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-podres\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.094408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.094232    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfczq\" (UniqueName: \"kubernetes.io/projected/cdc62982-b876-409c-9c0b-a2daa034bb0f-kube-api-access-lfczq\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.094408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.094269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-proc\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.094408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.094292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-lib-modules\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.094408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.094310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-sys\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.094408 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.094397    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-podres\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.094690 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.094398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-proc\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.094690 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.094404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-sys\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.094690 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.094451    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdc62982-b876-409c-9c0b-a2daa034bb0f-lib-modules\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.106901 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.106825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfczq\" (UniqueName: \"kubernetes.io/projected/cdc62982-b876-409c-9c0b-a2daa034bb0f-kube-api-access-lfczq\") pod \"perf-node-gather-daemonset-rhmp2\" (UID: \"cdc62982-b876-409c-9c0b-a2daa034bb0f\") " pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.220645 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.220604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:21.421993 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:21.421892    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"]
Jul 21 17:15:21.425598 ip-10-0-141-158 kubenswrapper[2580]: W0721 17:15:21.425570    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcdc62982_b876_409c_9c0b_a2daa034bb0f.slice/crio-93c722f0453a67b2f91e0266e012ab479a8ba06c167680b25dad031124c1e63d WatchSource:0}: Error finding container 93c722f0453a67b2f91e0266e012ab479a8ba06c167680b25dad031124c1e63d: Status 404 returned error can't find the container with id 93c722f0453a67b2f91e0266e012ab479a8ba06c167680b25dad031124c1e63d
Jul 21 17:15:22.269741 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:22.269700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2" event={"ID":"cdc62982-b876-409c-9c0b-a2daa034bb0f","Type":"ContainerStarted","Data":"2df456ffc95469a2b157c3509b6cb231befd8e5ba973018eb642d8d5d4c9acf4"}
Jul 21 17:15:22.270239 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:22.269763    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2" event={"ID":"cdc62982-b876-409c-9c0b-a2daa034bb0f","Type":"ContainerStarted","Data":"93c722f0453a67b2f91e0266e012ab479a8ba06c167680b25dad031124c1e63d"}
Jul 21 17:15:22.270239 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:22.269812    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:22.286820 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:22.286763    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2" podStartSLOduration=2.286744275 podStartE2EDuration="2.286744275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:15:22.286301772 +0000 UTC m=+2817.785663016" watchObservedRunningTime="2026-07-21 17:15:22.286744275 +0000 UTC m=+2817.786105522"
Jul 21 17:15:23.106097 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:23.106072    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2vgss_5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5/dns/0.log"
Jul 21 17:15:23.125754 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:23.125724    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2vgss_5edd3abf-bfb8-4ea8-aa78-6bbb5a4860f5/kube-rbac-proxy/0.log"
Jul 21 17:15:23.257566 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:23.257535    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qg6tx_e36e0695-ce82-44e8-89fc-689274f87243/dns-node-resolver/0.log"
Jul 21 17:15:23.871464 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:23.871433    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rkqfz_a5fbef28-96e8-46bd-84a7-aa7a08df42d2/node-ca/0.log"
Jul 21 17:15:25.025614 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:25.024801    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mk8pp_2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d/discovery/0.log"
Jul 21 17:15:25.071306 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:25.071281    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d77f95f8-tx4tj_cd7875a7-f877-4a29-aa5f-6f3dc41ebb35/kube-auth-proxy/0.log"
Jul 21 17:15:25.978389 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:25.978366    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nwcd9_590ad4b9-4708-4ced-9634-bb0791490700/serve-healthcheck-canary/0.log"
Jul 21 17:15:26.551389 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:26.551359    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kw7ld_8d41000f-6150-492d-b613-d1b08ef6e5b7/kube-rbac-proxy/0.log"
Jul 21 17:15:26.574069 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:26.574040    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kw7ld_8d41000f-6150-492d-b613-d1b08ef6e5b7/exporter/0.log"
Jul 21 17:15:26.599261 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:26.599233    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kw7ld_8d41000f-6150-492d-b613-d1b08ef6e5b7/extractor/0.log"
Jul 21 17:15:28.290374 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:28.290345    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nbkhg/perf-node-gather-daemonset-rhmp2"
Jul 21 17:15:29.425271 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:29.425230    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87c9445c4-n6cdx_aec37a82-46f6-46a3-8c24-37985db8b9d1/manager/0.log"
Jul 21 17:15:29.445080 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:29.445056    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87c9445c4-n6cdx_aec37a82-46f6-46a3-8c24-37985db8b9d1/copy-manifests/0.log"
Jul 21 17:15:30.886709 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:30.886634    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-54b99c46f6-k2r5d_9f774a1d-3ddb-45ce-81ca-6fbda981f33d/manager/0.log"
Jul 21 17:15:31.447953 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:31.447921    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mk8pp_2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d/discovery/0.log"
Jul 21 17:15:31.473774 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:31.473742    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d77f95f8-tx4tj_cd7875a7-f877-4a29-aa5f-6f3dc41ebb35/kube-auth-proxy/0.log"
Jul 21 17:15:32.995459 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:32.995430    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mk8pp_2c1d8c94-0b5d-45a2-a0d9-171ae1fa829d/discovery/0.log"
Jul 21 17:15:33.035175 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:33.035102    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d77f95f8-tx4tj_cd7875a7-f877-4a29-aa5f-6f3dc41ebb35/kube-auth-proxy/0.log"
Jul 21 17:15:38.150428 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:38.150405    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t2fh_b56ec3a5-eb74-4d28-b538-40c09c485e2b/kube-multus-additional-cni-plugins/0.log"
Jul 21 17:15:38.176414 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:38.176388    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t2fh_b56ec3a5-eb74-4d28-b538-40c09c485e2b/egress-router-binary-copy/0.log"
Jul 21 17:15:38.203566 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:38.203537    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t2fh_b56ec3a5-eb74-4d28-b538-40c09c485e2b/cni-plugins/0.log"
Jul 21 17:15:38.227161 ip-10-0-141-158 kubenswrapper[2580]: I0721 17:15:38.227142    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t2fh_b56ec3a5-eb74-4d28-b538-40c09c485e2b/bond-cni-plugin/0.log"