Jul 09 18:23:16.174430 ip-10-0-135-28 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 18:23:16.174442 ip-10-0-135-28 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 18:23:16.174449 ip-10-0-135-28 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 18:23:16.174659 ip-10-0-135-28 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 18:23:26.190644 ip-10-0-135-28 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 18:23:26.190666 ip-10-0-135-28 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9b0294809d684d5e941c201022379921 --
Jul 09 18:25:21.687100 ip-10-0-135-28 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.071276    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077768    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077779    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077783    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077787    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077790    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077793    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077796    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:25:22.164482 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077799    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:25:22.140459 ip-10-0-135-28 systemd[1]: Started Kubernetes Kubelet.
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077802    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077805    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077808    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077811    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077814    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077817    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077820    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077823    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077826    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077829    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077832    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077835    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077838    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077841    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077844    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077847    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077857    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077860    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077862    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:25:22.165424 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077879    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077882    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077885    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077887    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077890    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077893    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077897    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077902    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077905    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077908    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077911    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077914    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077917    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077919    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077922    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077925    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077927    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077930    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077932    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:25:22.166294 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077935    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077938    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077940    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077944    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077947    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077949    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077952    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077955    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077957    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077960    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077963    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077965    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077972    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077975    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077977    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077980    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077983    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077986    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077989    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077991    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:25:22.167181 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077994    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077997    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.077999    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078003    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078007    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078010    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078014    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078017    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078020    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078023    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078026    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078029    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078033    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078035    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078038    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078041    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078044    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078047    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078050    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:25:22.170611 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078052    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078056    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078060    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078063    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078066    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078069    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078071    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078074    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078077    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078080    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078082    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078085    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078087    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078090    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078781    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078788    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078791    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078794    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078797    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:25:22.172112 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078800    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078803    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078806    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078809    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078812    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078815    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078818    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078821    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078823    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078826    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078829    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078832    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078835    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078838    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078841    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078843    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078846    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078848    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078851    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:25:22.187839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078854    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078857    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078859    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078862    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078878    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078882    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078885    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078887    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078890    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078893    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078896    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078898    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078901    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078903    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078906    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078909    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078913    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078917    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078920    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078923    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:25:22.188650 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078925    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078928    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078931    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078934    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078937    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078941    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078945    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078948    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078951    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078954    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078957    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078960    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078963    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078965    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078967    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078970    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078973    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078975    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078978    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:25:22.189238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078982    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078984    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078987    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078991    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078995    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.078998    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079000    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079003    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079005    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079008    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079010    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079013    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079018    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079020    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079023    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079025    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079028    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079030    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079034    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079036    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:25:22.189854 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079039    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079042    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079045    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079047    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079050    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079053    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079055    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079058    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079061    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079066    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079069    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079072    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079074    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079077    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079080    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079082    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079427    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079432    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079434    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079437    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:25:22.190532 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079440    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079442    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079445    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079448    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079450    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079453    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079456    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079458    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079461    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079463    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079466    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079470    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079473    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079475    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079478    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079481    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079483    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079486    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079488    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079493    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:25:22.191505 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079496    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079501    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079504    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079506    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079509    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079512    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079515    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079518    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079521    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079524    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079526    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079529    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079532    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079535    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079537    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079540    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079542    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079545    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079547    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:25:22.192127 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079550    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079552    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079555    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079557    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079561    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079563    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079566    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079569    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079572    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079574    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079577    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079579    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079582    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079584    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079588    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079591    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079593    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079596    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079599    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079602    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:25:22.192970 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079604    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079607    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079609    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079612    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079614    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079617    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079619    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079621    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079624    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079626    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079630    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079634    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079636    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079639    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079642    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079645    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079648    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079651    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079653    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:25:22.193563 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079656    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079659    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079663    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079667    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079670    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079673    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079675    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079679    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079682    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079685    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079688    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079691    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079693    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079696    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079699    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079701    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.079704    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079785    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079794    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079815    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079822    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 18:25:22.194149 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079827    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079832    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079836    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079840    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079843    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079846    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079849    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079852    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079857    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079859    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079862    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079878    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079882    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079885    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079888    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079891    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079894    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079897    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079902    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079906    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079908    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079911    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079915    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079918    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 18:25:22.194708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079921    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079923    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079926    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079929    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079934    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079937    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079940    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079943    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079945    2566 flags.go:64] FLAG: --enable-server="true"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079948    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079952    2566 flags.go:64] FLAG: --event-burst="100"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079956    2566 flags.go:64] FLAG: --event-qps="50"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079959    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079962    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079965    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079968    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079972    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079975    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079978    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079981    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079983    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079987    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079990    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079992    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079996    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 18:25:22.195769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.079998    2566 flags.go:64] FLAG: --feature-gates=""
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080004    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080006    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080009    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080013    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080016    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080019    2566 flags.go:64] FLAG: --help="false"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080022    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-135-28.ec2.internal"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080025    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080028    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080030    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080033    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080037    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080040    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080042    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080045    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080048    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080051    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080054    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080057    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080059    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080062    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080065    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080069    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 18:25:22.196499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080072    2566 flags.go:64] FLAG: --lock-file=""
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080075    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080078    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080080    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080085    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080089    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080092    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080094    2566 flags.go:64] FLAG: --logging-format="text"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080097    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080101    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080104    2566 flags.go:64] FLAG: --manifest-url=""
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080107    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080111    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080114    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080118    2566 flags.go:64] FLAG: --max-pods="110"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080121    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080124    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080127    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080130    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080133    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080135    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080138    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080146    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080149    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080152    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 18:25:22.197188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080154    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080157    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080162    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080165    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080168    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080171    2566 flags.go:64] FLAG: --port="10250"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080174    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080178    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-035a7cfa62396779c"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080181    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080184    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080187    2566 flags.go:64] FLAG: --register-node="true"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080190    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080193    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080197    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080200    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080202    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080207    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080210    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080213    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080216    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080219    2566 flags.go:64] FLAG: --runonce="false"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080221    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080224    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080227    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080230    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 18:25:22.197823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080233    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080236    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080239    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080242    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080244    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080247    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080250    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080253    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080256    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080259    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080262    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080267    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080270    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080273    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080278    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080280    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080283    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080286    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080289    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080291    2566 flags.go:64] FLAG: --v="2"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080295    2566 flags.go:64] FLAG: --version="false"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080299    2566 flags.go:64] FLAG: --vmodule=""
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080302    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.080307    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080436    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:25:22.198534 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080440    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080443    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080446    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080449    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080452    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080454    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080457    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080459    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080462    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080465    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080467    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080470    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080472    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080475    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080477    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080480    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080482    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080485    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080488    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080490    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:25:22.199429 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080493    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080499    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080502    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080504    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080507    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080509    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080512    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080514    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080517    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080520    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080525    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080529    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080532    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080535    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080537    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080540    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080542    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080544    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080547    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:25:22.200244 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080549    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080552    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080555    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080557    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080560    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080562    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080565    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080568    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080570    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080573    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080575    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080578    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080580    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080584    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080590    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080594    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080597    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080600    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080603    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:25:22.200855 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080606    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080608    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080611    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080613    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080617    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080620    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080622    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080625    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080628    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080630    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080633    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080635    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080638    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080640    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080643    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080645    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080648    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080650    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080653    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080655    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:25:22.201455 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080658    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080661    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080663    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080666    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080668    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080671    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080673    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080677    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080680    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080682    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080685    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080688    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080690    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080693    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080696    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080698    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080702    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080705    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080708    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:25:22.398637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.080710    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.081171    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.087250    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.087264    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.087987    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.087995    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088048    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088052    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088056    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088060    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088064    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088067    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088071    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088073    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:25:22.412467 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088076    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088079    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088082    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088085    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088087    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088091    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088093    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088096    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088099    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088102    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088105    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088108    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088111    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088114    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088117    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088120    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088123    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088127    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088129    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:25:22.415017 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088132    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088138    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088142    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088145    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088150    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088154    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088158    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088162    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088166    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088170    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088174    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088176    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088179    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088183    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088186    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088189    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088191    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088194    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088196    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088199    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:25:22.415594 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088201    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088204    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088206    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088209    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088212    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088214    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088216    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088219    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088221    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088224    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088227    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088229    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088232    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088237    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088240    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088243    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088246    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088248    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088251    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088254    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:25:23.646617 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088256    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088259    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088262    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088264    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088267    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088270    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088272    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088275    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088278    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088280    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088283    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088285    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088287    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088290    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088293    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088295    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088298    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088300    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088303    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088305    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:25:23.647839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088308    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088310    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088313    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088316    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088318    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088322    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088324    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088327    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088330    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088333    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088336    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088338    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.088343    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088447    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:25:23.648490 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088452    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088455    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088458    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088461    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088464    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088467    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088470    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088473    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088475    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088478    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088480    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088483    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088486    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088489    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088491    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088495    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088499    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088502    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088505    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:25:23.648998 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088508    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088510    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088513    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088517    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088520    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088524    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088527    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088529    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088532    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088537    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088539    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088542    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088544    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088547    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088549    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088552    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088555    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088557    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088560    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088562    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:25:23.649506 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088565    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088568    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088570    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088573    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088575    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088578    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088580    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088583    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088586    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088588    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088590    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088593    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088595    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088598    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088600    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088603    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088605    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088609    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088612    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:25:23.650104 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088614    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088616    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088621    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088624    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088627    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088630    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088633    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088635    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088638    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088641    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088643    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088646    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088648    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088651    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088653    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088656    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088658    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088660    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088663    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088665    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:25:23.650614 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088668    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088671    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088673    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088675    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088678    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088680    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088683    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088686    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088688    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088690    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088694    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088697    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088699    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088702    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088706    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088709    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088711    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088714    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088716    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:25:23.651198 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:22.088719    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.088724    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.088832    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.091036    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.092456    2566 server.go:1019] "Starting client certificate rotation"
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.092550    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.092615    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.094605    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.100777    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.109883    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.111918    2566 log.go:25] "Validated CRI v1 image API"
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.112659    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 18:25:23.651747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.114839    2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 dd71c422-79c6-43e5-8309-8274853ad0e9:/dev/nvme0n1p3 f135587a-96e8-4b6e-b20a-084ddb6221ea:/dev/nvme0n1p4]
Jul 09 18:25:23.652162 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.114857    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 18:25:23.652232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.121006    2566 manager.go:216] Machine: {Timestamp:2026-07-09 18:25:22.119463369 +0000 UTC m=+0.288058584 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111480 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21adc7d909b1aef05f6b1cbb8328b7 SystemUUID:ec21adc7-d909-b1ae-f05f-6b1cbb8328b7 BootID:9b029480-9d68-4d5e-941c-201022379921 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:23:af:7b:2a:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:23:af:7b:2a:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:3f:2c:2d:0a:6f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 18:25:23.652232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.121119    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 18:25:23.652232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.121234    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 18:25:23.652232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.123474    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 18:25:23.652232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.123927    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 18:25:23.652232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.123948    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-28.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 18:25:23.652232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.125215    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.125226    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.125234    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.125761    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.127213    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.130584    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.130699    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.130725    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.130733    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.130742    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.130751    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.130758    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.131130    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.131151    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.133067    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.134999    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135034    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135204    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135219    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135227    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135235    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135245    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135252    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135572    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135580    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135588    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135596    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135616    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.135627    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.136384    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.136394    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.139734    2566 server.go:1268] "Started kubelet"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.139959    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.140011    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.140041    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.141783    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.144513    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-28.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.144548    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.144580    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 18:25:23.652518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.144586    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.145854    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.148735    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.148734    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.148827    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.148301    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-28.ec2.internal.18c0b25308e0f3af  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-28.ec2.internal,UID:ip-10-0-135-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-28.ec2.internal,},FirstTimestamp:2026-07-09 18:25:22.139706287 +0000 UTC m=+0.308301477,LastTimestamp:2026-07-09 18:25:22.139706287 +0000 UTC m=+0.308301477,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-28.ec2.internal,}"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149294    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149305    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149320    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149428    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149436    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149516    2566 factory.go:55] Registering systemd factory
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149549    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149747    2566 factory.go:153] Registering CRI-O factory
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149762    2566 factory.go:223] Registration of the crio container factory successfully
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149820    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149841    2566 factory.go:103] Registering Raw factory
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.149855    2566 manager.go:1196] Started watching for new ooms in manager
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.150233    2566 manager.go:349] Starting recovery of all containers
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.157210    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.157306    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.160330    2566 manager.go:354] Recovery completed
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.165103    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.166709    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.166727    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.166735    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.167459    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.167471    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.167488    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.169412    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.169706    2566 policy_none.go:49] "None policy: Start"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.169723    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 18:25:23.653580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.169745    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.169795    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-28.ec2.internal.18c0b2530a7d2513  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-28.ec2.internal,UID:ip-10-0-135-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-28.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-28.ec2.internal,},FirstTimestamp:2026-07-09 18:25:22.166719763 +0000 UTC m=+0.335314954,LastTimestamp:2026-07-09 18:25:22.166719763 +0000 UTC m=+0.335314954,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-28.ec2.internal,}"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.170557    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.170578    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.170590    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.170914    2566 policy_none.go:47] "Start"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.190862    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c2mrp"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.190893    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c2mrp"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.240366    2566 manager.go:337] "Starting Device Plugin manager"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.240392    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.240401    2566 server.go:87] "Starting device plugin registration server"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.240586    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.240597    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.240702    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.240787    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.240795    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.241185    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.241231    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.319050    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.320198    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.320226    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.320243    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.320270    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.322922    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.341210    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.341979    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.342002    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.342015    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.342040    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.352596    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.352612    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-28.ec2.internal\": node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:23.654479 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.394013    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.421037    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal"]
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.421094    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.422376    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.422401    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.422414    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.423666    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.423785    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.423808    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.424459    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.424479    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.424487    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.424459    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.424557    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.424570    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.425674    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.425694    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.426284    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.426303    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.426318    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.451821    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/14f0bfd5e98f4f5c1e726a24ae87af9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal\" (UID: \"14f0bfd5e98f4f5c1e726a24ae87af9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.451842    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14f0bfd5e98f4f5c1e726a24ae87af9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal\" (UID: \"14f0bfd5e98f4f5c1e726a24ae87af9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:22.451856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e502bf3c1df2bd74d5cf28e4fde27161-config\") pod \"kube-apiserver-proxy-ip-10-0-135-28.ec2.internal\" (UID: \"e502bf3c1df2bd74d5cf28e4fde27161\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.453401    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-28.ec2.internal\" not found" node="ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.457703    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-28.ec2.internal\" not found" node="ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.655355 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:22.494171    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:23.656096 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:23.585565    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/14f0bfd5e98f4f5c1e726a24ae87af9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal\" (UID: \"14f0bfd5e98f4f5c1e726a24ae87af9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.656096 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:23.585586    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14f0bfd5e98f4f5c1e726a24ae87af9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal\" (UID: \"14f0bfd5e98f4f5c1e726a24ae87af9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.656096 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:23.585601    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e502bf3c1df2bd74d5cf28e4fde27161-config\") pod \"kube-apiserver-proxy-ip-10-0-135-28.ec2.internal\" (UID: \"e502bf3c1df2bd74d5cf28e4fde27161\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.656096 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:23.585635    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e502bf3c1df2bd74d5cf28e4fde27161-config\") pod \"kube-apiserver-proxy-ip-10-0-135-28.ec2.internal\" (UID: \"e502bf3c1df2bd74d5cf28e4fde27161\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.656096 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:23.585675    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/14f0bfd5e98f4f5c1e726a24ae87af9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal\" (UID: \"14f0bfd5e98f4f5c1e726a24ae87af9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.656096 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:23.585683    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14f0bfd5e98f4f5c1e726a24ae87af9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal\" (UID: \"14f0bfd5e98f4f5c1e726a24ae87af9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.656096 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:23.627954    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:23.815283 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:23.728474    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:23.815283 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:23.787863    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.815283 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:23.793394    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:23.995296 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:23.829359    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:23.995296 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:23.929795    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:24.172943 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:24.030281    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:24.172943 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.118931    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:25:24.172943 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.124423    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 18:25:24.172943 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.124524    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 18:25:24.172943 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.124567    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 18:25:24.172943 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:24.130692    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:24.347416 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.199063    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:25:24.347416 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.226601    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 18:20:23 +0000 UTC" deadline="2028-01-13 12:59:28.386787064 +0000 UTC"
Jul 09 18:25:24.347416 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.226620    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13266h34m4.160169433s"
Jul 09 18:25:24.347416 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:24.231731    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:24.347416 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:24.332393    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:24.522045 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:24.433421    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:24.522045 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:24.448876    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode502bf3c1df2bd74d5cf28e4fde27161.slice/crio-999eb766b012aef2b3c79c85ed70a895a5c172c768569dd9fce2fda99b033dd2 WatchSource:0}: Error finding container 999eb766b012aef2b3c79c85ed70a895a5c172c768569dd9fce2fda99b033dd2: Status 404 returned error can't find the container with id 999eb766b012aef2b3c79c85ed70a895a5c172c768569dd9fce2fda99b033dd2
Jul 09 18:25:24.522045 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:24.449284    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f0bfd5e98f4f5c1e726a24ae87af9b.slice/crio-a566dd100f46ae1eca832cc130b71794decf466f34a57a3e1a040bc469b18c7f WatchSource:0}: Error finding container a566dd100f46ae1eca832cc130b71794decf466f34a57a3e1a040bc469b18c7f: Status 404 returned error can't find the container with id a566dd100f46ae1eca832cc130b71794decf466f34a57a3e1a040bc469b18c7f
Jul 09 18:25:24.522045 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.452982    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:25:24.723979 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:24.533869    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-28.ec2.internal\" not found"
Jul 09 18:25:24.723979 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.578716    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:25:24.723979 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.581670    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:24.723979 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.598027    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 18:25:24.723979 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.599157    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal"
Jul 09 18:25:24.723979 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.609256    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 18:25:25.048040 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:24.984839    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.164794    2566 apiserver.go:52] "Watching apiserver"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.177734    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.178228    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal","openshift-multus/multus-additional-cni-plugins-6zms9","openshift-multus/multus-rmknh","openshift-multus/network-metrics-daemon-llr8j","openshift-network-diagnostics/network-check-target-xx64v","openshift-ovn-kubernetes/ovnkube-node-chbbq","kube-system/global-pull-secret-syncer-7m46n","kube-system/konnectivity-agent-9wx8q","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn","openshift-cluster-node-tuning-operator/tuned-v2wcw","openshift-dns/node-resolver-2mb6q","openshift-network-operator/iptables-alerter-rjbtr","kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal","openshift-image-registry/node-ca-slptj"]
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.179685    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.179755    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.180926    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.182337    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.182507    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.183303    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.183368    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.184340    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.184389    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.185900    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.187015    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.187104    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.187267    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.187340    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.187348    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bdzsk\""
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.187267    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h2mrl\""
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.187503    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.188184    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.189276    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.189898    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.190333    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 18:25:25.192506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.190732    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.194919 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.191967    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 18:25:25.194919 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.192029    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 18:25:25.194919 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.192235    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.194919 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.193420    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 18:25:25.194919 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.193636    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 18:25:25.194919 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.193641    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m4wfl\""
Jul 09 18:25:25.194919 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.193777    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197061    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197129    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5mfs6\""
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197166    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197207    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197426    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-system-cni-dir\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197464    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-cnibin\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197495    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-daemon-config\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197529    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-run-multus-certs\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59lbn\" (UniqueName: \"kubernetes.io/projected/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-kube-api-access-59lbn\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197598    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197601    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197686    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59b49275-d607-420f-acaf-10d3b9843b44-hosts-file\") pod \"node-resolver-2mb6q\" (UID: \"59b49275-d607-420f-acaf-10d3b9843b44\") " pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197718    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59b49275-d607-420f-acaf-10d3b9843b44-tmp-dir\") pod \"node-resolver-2mb6q\" (UID: \"59b49275-d607-420f-acaf-10d3b9843b44\") " pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197762    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-os-release\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197801    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-socket-dir-parent\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197841    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-run-k8s-cni-cncf-io\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.199497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197897    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-var-lib-cni-bin\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.197928    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-var-lib-kubelet\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.198153    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-etc-kubernetes\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.198174    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.198179    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9876487f-ee1d-4875-940e-d0032bc17427-kubelet-config\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.199443    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9876487f-ee1d-4875-940e-d0032bc17427-dbus\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.199481    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-cni-dir\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.199509    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-cni-binary-copy\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.199540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgzmz\" (UniqueName: \"kubernetes.io/projected/735bceb1-4f3b-4163-a164-a70f462b5aac-kube-api-access-mgzmz\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.199570    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjf92\" (UniqueName: \"kubernetes.io/projected/59b49275-d607-420f-acaf-10d3b9843b44-kube-api-access-wjf92\") pod \"node-resolver-2mb6q\" (UID: \"59b49275-d607-420f-acaf-10d3b9843b44\") " pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.199601    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-run-netns\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.199630    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-var-lib-cni-multus\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.199678    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-hostroot\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.199709    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-conf-dir\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.201512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.199750    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:25.204285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.203276    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:25:25.204285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.203363    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 18:25:25.204285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.203457    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 18:25:25.204285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.203696    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 18:25:25.204285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.203733    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 18:25:25.204285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.203830    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-585g8\""
Jul 09 18:25:25.204285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.203900    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:25:25.204285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.204255    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 18:25:25.204285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.204278    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 18:25:25.206680 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.205484    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 18:25:25.206680 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.205683    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 18:25:25.206680 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.205747    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7zgpl\""
Jul 09 18:25:25.206680 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.205877    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2jvbp\""
Jul 09 18:25:25.206680 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.205987    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-258kw\""
Jul 09 18:25:25.206680 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.206004    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r4cnv\""
Jul 09 18:25:25.206680 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.206121    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 18:25:25.206680 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.206461    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 18:25:25.275609 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.222606    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 18:25:25.275609 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.247959    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r9kj8"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.280805    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r9kj8"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.283178    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.299876    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-cni-binary-copy\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.299909    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgzmz\" (UniqueName: \"kubernetes.io/projected/735bceb1-4f3b-4163-a164-a70f462b5aac-kube-api-access-mgzmz\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.299940    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-run-ovn-kubernetes\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.299966    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.299992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-socket-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300032    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-sys-fs\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300055    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-daemon-config\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300081    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjf92\" (UniqueName: \"kubernetes.io/projected/59b49275-d607-420f-acaf-10d3b9843b44-kube-api-access-wjf92\") pod \"node-resolver-2mb6q\" (UID: \"59b49275-d607-420f-acaf-10d3b9843b44\") " pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300106    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-run-netns\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300127    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-hostroot\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300149    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-cni-netd\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300175    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64f7b4ed-439b-408d-9b3b-cb169308ab35-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.368608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300198    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-modprobe-d\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300221    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-sysconfig\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300258    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-cni-bin\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300287    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ab113693-b417-46df-895a-91f05883b294-ovnkube-config\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300309    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-cnibin\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300331    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-systemd\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300345    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjmqv\" (UniqueName: \"kubernetes.io/projected/f98f059e-caca-400d-b05c-5d18371deac1-kube-api-access-bjmqv\") pod \"iptables-alerter-rjbtr\" (UID: \"f98f059e-caca-400d-b05c-5d18371deac1\") " pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300362    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-socket-dir-parent\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300385    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-var-lib-cni-bin\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300399    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-run-netns\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300415    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qkml\" (UniqueName: \"kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml\") pod \"network-check-target-xx64v\" (UID: \"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0\") " pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300448    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bb32097-7550-49c6-86fa-5d8248ace642-agent-certs\") pod \"konnectivity-agent-9wx8q\" (UID: \"2bb32097-7550-49c6-86fa-5d8248ace642\") " pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bb32097-7550-49c6-86fa-5d8248ace642-konnectivity-ca\") pod \"konnectivity-agent-9wx8q\" (UID: \"2bb32097-7550-49c6-86fa-5d8248ace642\") " pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300509    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-node-log\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.371397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300562    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-cni-binary-copy\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300630    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-socket-dir-parent\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300674    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-var-lib-cni-bin\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300680    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ab113693-b417-46df-895a-91f05883b294-env-overrides\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300722    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-run-netns\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300743    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-etc-selinux\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300757    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-hostroot\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300775    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64f7b4ed-439b-408d-9b3b-cb169308ab35-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300779    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-daemon-config\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300810    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8q2b\" (UniqueName: \"kubernetes.io/projected/64f7b4ed-439b-408d-9b3b-cb169308ab35-kube-api-access-f8q2b\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300834    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-sysctl-d\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300858    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-lib-modules\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzb2j\" (UniqueName: \"kubernetes.io/projected/67607ff1-affa-4647-965e-119f2a24637f-kube-api-access-gzb2j\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300912    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-conf-dir\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.372042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300943    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ab113693-b417-46df-895a-91f05883b294-ovnkube-script-lib\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300963    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-conf-dir\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.300967    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-registration-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301000    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-device-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301027    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-system-cni-dir\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301051    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-cnibin\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301090    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301091    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-system-cni-dir\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301117    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-kubelet\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301152    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-cnibin\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301178    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64f7b4ed-439b-408d-9b3b-cb169308ab35-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301201    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-kubernetes\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.301281    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301310    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67607ff1-affa-4647-965e-119f2a24637f-tmp\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.442923 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.301351    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs podName:735bceb1-4f3b-4163-a164-a70f462b5aac nodeName:}" failed. No retries permitted until 2026-07-09 18:25:25.801324926 +0000 UTC m=+2.937142376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs") pod "network-metrics-daemon-llr8j" (UID: "735bceb1-4f3b-4163-a164-a70f462b5aac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301385    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-etc-kubernetes\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301418    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-slash\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9876487f-ee1d-4875-940e-d0032bc17427-kubelet-config\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301462    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9d4d\" (UniqueName: \"kubernetes.io/projected/90c26f7b-938e-4b61-972c-8e1ff90c4b08-kube-api-access-l9d4d\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301470    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-etc-kubernetes\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301526    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9876487f-ee1d-4875-940e-d0032bc17427-kubelet-config\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07a65c98-48c0-4cd8-a737-5908696e29b6-serviceca\") pod \"node-ca-slptj\" (UID: \"07a65c98-48c0-4cd8-a737-5908696e29b6\") " pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301587    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f98f059e-caca-400d-b05c-5d18371deac1-iptables-alerter-script\") pod \"iptables-alerter-rjbtr\" (UID: \"f98f059e-caca-400d-b05c-5d18371deac1\") " pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301615    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59lbn\" (UniqueName: \"kubernetes.io/projected/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-kube-api-access-59lbn\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301655    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-var-lib-cni-multus\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301682    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-run-openvswitch\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301696    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-var-lib-cni-multus\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301706    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.443952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301729    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/67607ff1-affa-4647-965e-119f2a24637f-etc-tuned\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301750    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07a65c98-48c0-4cd8-a737-5908696e29b6-host\") pod \"node-ca-slptj\" (UID: \"07a65c98-48c0-4cd8-a737-5908696e29b6\") " pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-systemd-units\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301830    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-etc-openvswitch\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.301885    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-system-cni-dir\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-sysctl-conf\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.301929    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret podName:9876487f-ee1d-4875-940e-d0032bc17427 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:25.801914719 +0000 UTC m=+2.937732180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret") pod "global-pull-secret-syncer-7m46n" (UID: "9876487f-ee1d-4875-940e-d0032bc17427") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301961    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-run\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301981    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-sys\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.301995    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdrvh\" (UniqueName: \"kubernetes.io/projected/07a65c98-48c0-4cd8-a737-5908696e29b6-kube-api-access-cdrvh\") pod \"node-ca-slptj\" (UID: \"07a65c98-48c0-4cd8-a737-5908696e29b6\") " pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302013    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59b49275-d607-420f-acaf-10d3b9843b44-hosts-file\") pod \"node-resolver-2mb6q\" (UID: \"59b49275-d607-420f-acaf-10d3b9843b44\") " pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302029    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59b49275-d607-420f-acaf-10d3b9843b44-tmp-dir\") pod \"node-resolver-2mb6q\" (UID: \"59b49275-d607-420f-acaf-10d3b9843b44\") " pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.444565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302074    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-os-release\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302088    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59b49275-d607-420f-acaf-10d3b9843b44-hosts-file\") pod \"node-resolver-2mb6q\" (UID: \"59b49275-d607-420f-acaf-10d3b9843b44\") " pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302107    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-var-lib-kubelet\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302132    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f98f059e-caca-400d-b05c-5d18371deac1-host-slash\") pod \"iptables-alerter-rjbtr\" (UID: \"f98f059e-caca-400d-b05c-5d18371deac1\") " pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302155    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-os-release\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302159    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9876487f-ee1d-4875-940e-d0032bc17427-dbus\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-cni-dir\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302221    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-run-ovn\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302247    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302255    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59b49275-d607-420f-acaf-10d3b9843b44-tmp-dir\") pod \"node-resolver-2mb6q\" (UID: \"59b49275-d607-420f-acaf-10d3b9843b44\") " pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302270    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-os-release\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302305    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-var-lib-openvswitch\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302318    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-multus-cni-dir\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302331    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-log-socket\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.445601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302350    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh56m\" (UniqueName: \"kubernetes.io/projected/ab113693-b417-46df-895a-91f05883b294-kube-api-access-jh56m\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302351    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9876487f-ee1d-4875-940e-d0032bc17427-dbus\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302382    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-run-multus-certs\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302406    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-run-systemd\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302420    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-host\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302437    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-run-k8s-cni-cncf-io\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-var-lib-kubelet\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302464    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-run-multus-certs\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302490    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-run-k8s-cni-cncf-io\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302509    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ab113693-b417-46df-895a-91f05883b294-ovn-node-metrics-cert\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.302545    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-host-var-lib-kubelet\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.325342    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.329315    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgzmz\" (UniqueName: \"kubernetes.io/projected/735bceb1-4f3b-4163-a164-a70f462b5aac-kube-api-access-mgzmz\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.329315    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59lbn\" (UniqueName: \"kubernetes.io/projected/0fbe583d-3a9d-419b-867d-4a0c1f4f0580-kube-api-access-59lbn\") pod \"multus-rmknh\" (UID: \"0fbe583d-3a9d-419b-867d-4a0c1f4f0580\") " pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.335492    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjf92\" (UniqueName: \"kubernetes.io/projected/59b49275-d607-420f-acaf-10d3b9843b44-kube-api-access-wjf92\") pod \"node-resolver-2mb6q\" (UID: \"59b49275-d607-420f-acaf-10d3b9843b44\") " pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.446229 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.358021    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal" event={"ID":"e502bf3c1df2bd74d5cf28e4fde27161","Type":"ContainerStarted","Data":"999eb766b012aef2b3c79c85ed70a895a5c172c768569dd9fce2fda99b033dd2"}
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.359394    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal" event={"ID":"14f0bfd5e98f4f5c1e726a24ae87af9b","Type":"ContainerStarted","Data":"a566dd100f46ae1eca832cc130b71794decf466f34a57a3e1a040bc469b18c7f"}
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.402746    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-run-ovn\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.402775    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.402796    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-os-release\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.402817    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-var-lib-openvswitch\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.402987    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-log-socket\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403143    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh56m\" (UniqueName: \"kubernetes.io/projected/ab113693-b417-46df-895a-91f05883b294-kube-api-access-jh56m\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.402850    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-run-ovn\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403177    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-run-systemd\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403218    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-host\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403216    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403250    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ab113693-b417-46df-895a-91f05883b294-ovn-node-metrics-cert\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403277    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-run-ovn-kubernetes\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403288    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-os-release\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403365    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403416    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-log-socket\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403708    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-run-systemd\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403766    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-host\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403925    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-run-ovn-kubernetes\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.403964    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-var-lib-openvswitch\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404053    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404099    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-socket-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404136    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-sys-fs\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404173    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-cni-netd\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64f7b4ed-439b-408d-9b3b-cb169308ab35-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404229    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-modprobe-d\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404259    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-sysconfig\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.447800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404289    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-cni-bin\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ab113693-b417-46df-895a-91f05883b294-ovnkube-config\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404353    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-cnibin\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404377    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-systemd\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404411    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjmqv\" (UniqueName: \"kubernetes.io/projected/f98f059e-caca-400d-b05c-5d18371deac1-kube-api-access-bjmqv\") pod \"iptables-alerter-rjbtr\" (UID: \"f98f059e-caca-400d-b05c-5d18371deac1\") " pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404444    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-run-netns\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404476    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qkml\" (UniqueName: \"kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml\") pod \"network-check-target-xx64v\" (UID: \"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0\") " pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404501    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bb32097-7550-49c6-86fa-5d8248ace642-agent-certs\") pod \"konnectivity-agent-9wx8q\" (UID: \"2bb32097-7550-49c6-86fa-5d8248ace642\") " pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404530    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bb32097-7550-49c6-86fa-5d8248ace642-konnectivity-ca\") pod \"konnectivity-agent-9wx8q\" (UID: \"2bb32097-7550-49c6-86fa-5d8248ace642\") " pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404560    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-node-log\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404592    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ab113693-b417-46df-895a-91f05883b294-env-overrides\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404623    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-etc-selinux\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404666    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64f7b4ed-439b-408d-9b3b-cb169308ab35-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404699    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8q2b\" (UniqueName: \"kubernetes.io/projected/64f7b4ed-439b-408d-9b3b-cb169308ab35-kube-api-access-f8q2b\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.448868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404730    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-sysctl-d\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-lib-modules\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404806    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzb2j\" (UniqueName: \"kubernetes.io/projected/67607ff1-affa-4647-965e-119f2a24637f-kube-api-access-gzb2j\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404836    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ab113693-b417-46df-895a-91f05883b294-ovnkube-script-lib\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404866    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-registration-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404898    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-device-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404947    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-kubelet\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.404987    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64f7b4ed-439b-408d-9b3b-cb169308ab35-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.405120    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-etc-selinux\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.405698    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64f7b4ed-439b-408d-9b3b-cb169308ab35-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.405802    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-run-netns\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.405975    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-kubernetes\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406013    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67607ff1-affa-4647-965e-119f2a24637f-tmp\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406044    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-slash\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.450554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406095    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-slash\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-node-log\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406171    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-sysctl-d\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406215    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9d4d\" (UniqueName: \"kubernetes.io/projected/90c26f7b-938e-4b61-972c-8e1ff90c4b08-kube-api-access-l9d4d\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406253    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07a65c98-48c0-4cd8-a737-5908696e29b6-serviceca\") pod \"node-ca-slptj\" (UID: \"07a65c98-48c0-4cd8-a737-5908696e29b6\") " pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406285    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f98f059e-caca-400d-b05c-5d18371deac1-iptables-alerter-script\") pod \"iptables-alerter-rjbtr\" (UID: \"f98f059e-caca-400d-b05c-5d18371deac1\") " pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-run-openvswitch\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406349    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-sys-fs\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406362    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406371    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-modprobe-d\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406411    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-cni-netd\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406435    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-sysconfig\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406481    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-lib-modules\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406512    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-cni-bin\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.451317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406512    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-socket-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406536    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/67607ff1-affa-4647-965e-119f2a24637f-etc-tuned\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406597    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07a65c98-48c0-4cd8-a737-5908696e29b6-host\") pod \"node-ca-slptj\" (UID: \"07a65c98-48c0-4cd8-a737-5908696e29b6\") " pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406620    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406599    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64f7b4ed-439b-408d-9b3b-cb169308ab35-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406677    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ab113693-b417-46df-895a-91f05883b294-ovn-node-metrics-cert\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406719    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-kubernetes\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406739    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-registration-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406797    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-systemd\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406842    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07a65c98-48c0-4cd8-a737-5908696e29b6-host\") pod \"node-ca-slptj\" (UID: \"07a65c98-48c0-4cd8-a737-5908696e29b6\") " pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406899    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90c26f7b-938e-4b61-972c-8e1ff90c4b08-device-dir\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.406956    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-host-kubelet\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407072    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bb32097-7550-49c6-86fa-5d8248ace642-konnectivity-ca\") pod \"konnectivity-agent-9wx8q\" (UID: \"2bb32097-7550-49c6-86fa-5d8248ace642\") " pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-systemd-units\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407156    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-etc-openvswitch\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.451990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407184    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-system-cni-dir\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-sysctl-conf\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407237    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-run\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-sys\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407291    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdrvh\" (UniqueName: \"kubernetes.io/projected/07a65c98-48c0-4cd8-a737-5908696e29b6-kube-api-access-cdrvh\") pod \"node-ca-slptj\" (UID: \"07a65c98-48c0-4cd8-a737-5908696e29b6\") " pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-var-lib-kubelet\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407391    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f98f059e-caca-400d-b05c-5d18371deac1-host-slash\") pod \"iptables-alerter-rjbtr\" (UID: \"f98f059e-caca-400d-b05c-5d18371deac1\") " pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407505    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f98f059e-caca-400d-b05c-5d18371deac1-host-slash\") pod \"iptables-alerter-rjbtr\" (UID: \"f98f059e-caca-400d-b05c-5d18371deac1\") " pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407555    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-systemd-units\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407598    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-etc-openvswitch\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407632    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-cnibin\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407662    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64f7b4ed-439b-408d-9b3b-cb169308ab35-system-cni-dir\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407778    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-etc-sysctl-conf\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.408019    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-run\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.408073    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-sys\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.452714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.408110    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07a65c98-48c0-4cd8-a737-5908696e29b6-serviceca\") pod \"node-ca-slptj\" (UID: \"07a65c98-48c0-4cd8-a737-5908696e29b6\") " pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.408301    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67607ff1-affa-4647-965e-119f2a24637f-var-lib-kubelet\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.407366    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ab113693-b417-46df-895a-91f05883b294-env-overrides\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.408536    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64f7b4ed-439b-408d-9b3b-cb169308ab35-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.408603    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab113693-b417-46df-895a-91f05883b294-run-openvswitch\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.408682    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f98f059e-caca-400d-b05c-5d18371deac1-iptables-alerter-script\") pod \"iptables-alerter-rjbtr\" (UID: \"f98f059e-caca-400d-b05c-5d18371deac1\") " pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.409078    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67607ff1-affa-4647-965e-119f2a24637f-tmp\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.409347    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bb32097-7550-49c6-86fa-5d8248ace642-agent-certs\") pod \"konnectivity-agent-9wx8q\" (UID: \"2bb32097-7550-49c6-86fa-5d8248ace642\") " pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.409894    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ab113693-b417-46df-895a-91f05883b294-ovnkube-script-lib\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.409990    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ab113693-b417-46df-895a-91f05883b294-ovnkube-config\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.412014    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/67607ff1-affa-4647-965e-119f2a24637f-etc-tuned\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.424435    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh56m\" (UniqueName: \"kubernetes.io/projected/ab113693-b417-46df-895a-91f05883b294-kube-api-access-jh56m\") pod \"ovnkube-node-chbbq\" (UID: \"ab113693-b417-46df-895a-91f05883b294\") " pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.450119    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8q2b\" (UniqueName: \"kubernetes.io/projected/64f7b4ed-439b-408d-9b3b-cb169308ab35-kube-api-access-f8q2b\") pod \"multus-additional-cni-plugins-6zms9\" (UID: \"64f7b4ed-439b-408d-9b3b-cb169308ab35\") " pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.450485    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.450503    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:25:25.453435 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.450516    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2qkml for pod openshift-network-diagnostics/network-check-target-xx64v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:25.453962 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.450572    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml podName:962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:25.950556384 +0000 UTC m=+3.086373848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2qkml" (UniqueName: "kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml") pod "network-check-target-xx64v" (UID: "962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:25.453962 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.450818    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjmqv\" (UniqueName: \"kubernetes.io/projected/f98f059e-caca-400d-b05c-5d18371deac1-kube-api-access-bjmqv\") pod \"iptables-alerter-rjbtr\" (UID: \"f98f059e-caca-400d-b05c-5d18371deac1\") " pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.453962 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.452419    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdrvh\" (UniqueName: \"kubernetes.io/projected/07a65c98-48c0-4cd8-a737-5908696e29b6-kube-api-access-cdrvh\") pod \"node-ca-slptj\" (UID: \"07a65c98-48c0-4cd8-a737-5908696e29b6\") " pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.453962 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.453276    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzb2j\" (UniqueName: \"kubernetes.io/projected/67607ff1-affa-4647-965e-119f2a24637f-kube-api-access-gzb2j\") pod \"tuned-v2wcw\" (UID: \"67607ff1-affa-4647-965e-119f2a24637f\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.454179 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.454160    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9d4d\" (UniqueName: \"kubernetes.io/projected/90c26f7b-938e-4b61-972c-8e1ff90c4b08-kube-api-access-l9d4d\") pod \"aws-ebs-csi-driver-node-mbdhn\" (UID: \"90c26f7b-938e-4b61-972c-8e1ff90c4b08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.493176 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.493150    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2mb6q"
Jul 09 18:25:25.501680 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.501556    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rmknh"
Jul 09 18:25:25.504374 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:25.504346    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59b49275_d607_420f_acaf_10d3b9843b44.slice/crio-fe2d0918aa55b33acd866fdbb37c7362dfd3bc5e6dcc88219371251ed5901112 WatchSource:0}: Error finding container fe2d0918aa55b33acd866fdbb37c7362dfd3bc5e6dcc88219371251ed5901112: Status 404 returned error can't find the container with id fe2d0918aa55b33acd866fdbb37c7362dfd3bc5e6dcc88219371251ed5901112
Jul 09 18:25:25.511274 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.511255    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:25.514078 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:25.514056    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fbe583d_3a9d_419b_867d_4a0c1f4f0580.slice/crio-480d78ecbdbdd6835ec4354ddb86c296e86071de02079ff0d4bcd307e0b5fecb WatchSource:0}: Error finding container 480d78ecbdbdd6835ec4354ddb86c296e86071de02079ff0d4bcd307e0b5fecb: Status 404 returned error can't find the container with id 480d78ecbdbdd6835ec4354ddb86c296e86071de02079ff0d4bcd307e0b5fecb
Jul 09 18:25:25.515560 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.515368    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn"
Jul 09 18:25:25.521751 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:25.521718    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab113693_b417_46df_895a_91f05883b294.slice/crio-67de9b5dcf52e41c339b64a36e784624579721c3391521700805bbba2f486919 WatchSource:0}: Error finding container 67de9b5dcf52e41c339b64a36e784624579721c3391521700805bbba2f486919: Status 404 returned error can't find the container with id 67de9b5dcf52e41c339b64a36e784624579721c3391521700805bbba2f486919
Jul 09 18:25:25.522766 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.522749    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-slptj"
Jul 09 18:25:25.531337 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.531318    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6zms9"
Jul 09 18:25:25.531767 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:25.531737    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07a65c98_48c0_4cd8_a737_5908696e29b6.slice/crio-01bc9eaacb1c4eaa18ef3102a9ebc5b3ca216267a1f0d4f4097037d9477b589e WatchSource:0}: Error finding container 01bc9eaacb1c4eaa18ef3102a9ebc5b3ca216267a1f0d4f4097037d9477b589e: Status 404 returned error can't find the container with id 01bc9eaacb1c4eaa18ef3102a9ebc5b3ca216267a1f0d4f4097037d9477b589e
Jul 09 18:25:25.538720 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.538688    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v2wcw"
Jul 09 18:25:25.541717 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:25.541602    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f7b4ed_439b_408d_9b3b_cb169308ab35.slice/crio-9e6225ad370cc80f080b0c49651d0cf72ead5f07c2edb292e802fb5181951d0a WatchSource:0}: Error finding container 9e6225ad370cc80f080b0c49651d0cf72ead5f07c2edb292e802fb5181951d0a: Status 404 returned error can't find the container with id 9e6225ad370cc80f080b0c49651d0cf72ead5f07c2edb292e802fb5181951d0a
Jul 09 18:25:25.544126 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.544109    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:25.551381 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:25.551358    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67607ff1_affa_4647_965e_119f2a24637f.slice/crio-7eba56b4a03fccb7949e32d8df4973d337804ba043c97b76a649d102c5303bfb WatchSource:0}: Error finding container 7eba56b4a03fccb7949e32d8df4973d337804ba043c97b76a649d102c5303bfb: Status 404 returned error can't find the container with id 7eba56b4a03fccb7949e32d8df4973d337804ba043c97b76a649d102c5303bfb
Jul 09 18:25:25.554640 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:25.554612    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb32097_7550_49c6_86fa_5d8248ace642.slice/crio-fc0624ed5a23d65ecd6629555905fc43f1547add59346576acf865b3086774a6 WatchSource:0}: Error finding container fc0624ed5a23d65ecd6629555905fc43f1547add59346576acf865b3086774a6: Status 404 returned error can't find the container with id fc0624ed5a23d65ecd6629555905fc43f1547add59346576acf865b3086774a6
Jul 09 18:25:25.579256 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.579116    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rjbtr"
Jul 09 18:25:25.588013 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:25:25.587989    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf98f059e_caca_400d_b05c_5d18371deac1.slice/crio-c5732572983ccfd588eecd7273893564bdfddc58015042387ae4c1f425a33bd9 WatchSource:0}: Error finding container c5732572983ccfd588eecd7273893564bdfddc58015042387ae4c1f425a33bd9: Status 404 returned error can't find the container with id c5732572983ccfd588eecd7273893564bdfddc58015042387ae4c1f425a33bd9
Jul 09 18:25:25.810884 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.810811    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:25.811037 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:25.810916    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:25.811037 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.810945    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:25.811037 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.811010    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret podName:9876487f-ee1d-4875-940e-d0032bc17427 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:26.810991579 +0000 UTC m=+3.946809034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret") pod "global-pull-secret-syncer-7m46n" (UID: "9876487f-ee1d-4875-940e-d0032bc17427") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:25.811037 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.811018    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:25.811239 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:25.811067    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs podName:735bceb1-4f3b-4163-a164-a70f462b5aac nodeName:}" failed. No retries permitted until 2026-07-09 18:25:26.811051323 +0000 UTC m=+3.946868787 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs") pod "network-metrics-daemon-llr8j" (UID: "735bceb1-4f3b-4163-a164-a70f462b5aac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:26.013236 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.013208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qkml\" (UniqueName: \"kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml\") pod \"network-check-target-xx64v\" (UID: \"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0\") " pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:26.013438 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:26.013382    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:25:26.013438 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:26.013405    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:25:26.013438 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:26.013420    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2qkml for pod openshift-network-diagnostics/network-check-target-xx64v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:26.013562 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:26.013475    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml podName:962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:27.013460097 +0000 UTC m=+4.149277556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qkml" (UniqueName: "kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml") pod "network-check-target-xx64v" (UID: "962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:26.283090 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.283054    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 18:20:25 +0000 UTC" deadline="2028-04-11 14:23:39.26068916 +0000 UTC"
Jul 09 18:25:26.283090 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.283081    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15403h58m12.977611614s"
Jul 09 18:25:26.362110 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.362073    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rjbtr" event={"ID":"f98f059e-caca-400d-b05c-5d18371deac1","Type":"ContainerStarted","Data":"c5732572983ccfd588eecd7273893564bdfddc58015042387ae4c1f425a33bd9"}
Jul 09 18:25:26.363350 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.363323    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9wx8q" event={"ID":"2bb32097-7550-49c6-86fa-5d8248ace642","Type":"ContainerStarted","Data":"fc0624ed5a23d65ecd6629555905fc43f1547add59346576acf865b3086774a6"}
Jul 09 18:25:26.364438 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.364410    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn" event={"ID":"90c26f7b-938e-4b61-972c-8e1ff90c4b08","Type":"ContainerStarted","Data":"d2b4ec233bf5dc1d85a7ca3788ccaf99bc46b0a2681746e0b63d6a64fa7ddb3b"}
Jul 09 18:25:26.365630 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.365606    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rmknh" event={"ID":"0fbe583d-3a9d-419b-867d-4a0c1f4f0580","Type":"ContainerStarted","Data":"480d78ecbdbdd6835ec4354ddb86c296e86071de02079ff0d4bcd307e0b5fecb"}
Jul 09 18:25:26.366804 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.366775    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v2wcw" event={"ID":"67607ff1-affa-4647-965e-119f2a24637f","Type":"ContainerStarted","Data":"7eba56b4a03fccb7949e32d8df4973d337804ba043c97b76a649d102c5303bfb"}
Jul 09 18:25:26.367922 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.367897    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zms9" event={"ID":"64f7b4ed-439b-408d-9b3b-cb169308ab35","Type":"ContainerStarted","Data":"9e6225ad370cc80f080b0c49651d0cf72ead5f07c2edb292e802fb5181951d0a"}
Jul 09 18:25:26.368997 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.368969    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-slptj" event={"ID":"07a65c98-48c0-4cd8-a737-5908696e29b6","Type":"ContainerStarted","Data":"01bc9eaacb1c4eaa18ef3102a9ebc5b3ca216267a1f0d4f4097037d9477b589e"}
Jul 09 18:25:26.370113 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.370088    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq" event={"ID":"ab113693-b417-46df-895a-91f05883b294","Type":"ContainerStarted","Data":"67de9b5dcf52e41c339b64a36e784624579721c3391521700805bbba2f486919"}
Jul 09 18:25:26.371166 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.371145    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2mb6q" event={"ID":"59b49275-d607-420f-acaf-10d3b9843b44","Type":"ContainerStarted","Data":"fe2d0918aa55b33acd866fdbb37c7362dfd3bc5e6dcc88219371251ed5901112"}
Jul 09 18:25:26.820072 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.819556    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:26.820072 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:26.819730    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:26.820072 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:26.819751    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:26.820072 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:26.819793    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs podName:735bceb1-4f3b-4163-a164-a70f462b5aac nodeName:}" failed. No retries permitted until 2026-07-09 18:25:28.819774671 +0000 UTC m=+5.955592140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs") pod "network-metrics-daemon-llr8j" (UID: "735bceb1-4f3b-4163-a164-a70f462b5aac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:26.820072 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:26.819845    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:26.820072 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:26.819889    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret podName:9876487f-ee1d-4875-940e-d0032bc17427 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:28.819874718 +0000 UTC m=+5.955692180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret") pod "global-pull-secret-syncer-7m46n" (UID: "9876487f-ee1d-4875-940e-d0032bc17427") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:27.021510 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.021403    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qkml\" (UniqueName: \"kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml\") pod \"network-check-target-xx64v\" (UID: \"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0\") " pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:27.021670 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:27.021563    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:25:27.021670 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:27.021584    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:25:27.021670 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:27.021596    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2qkml for pod openshift-network-diagnostics/network-check-target-xx64v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:27.021670 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:27.021664    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml podName:962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:29.021630602 +0000 UTC m=+6.157448062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qkml" (UniqueName: "kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml") pod "network-check-target-xx64v" (UID: "962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:27.184067 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.183775    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:25:27.283965 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.283866    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 18:20:25 +0000 UTC" deadline="2028-03-26 09:43:55.083025492 +0000 UTC"
Jul 09 18:25:27.283965 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.283920    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15015h18m27.799108408s"
Jul 09 18:25:27.356674 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.356230    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:27.356674 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:27.356362    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:27.356870 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.356758    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:27.356870 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:27.356844    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:27.356989 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.356902    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:27.356989 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:27.356971    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:27.385541 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.385512    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal" event={"ID":"e502bf3c1df2bd74d5cf28e4fde27161","Type":"ContainerStarted","Data":"d3f1f87e766dda56e23a8fda397f3d9158f654cc18743919f302cc647cc7ebe4"}
Jul 09 18:25:27.389987 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.389962    2566 generic.go:356] "Generic (PLEG): container finished" podID="14f0bfd5e98f4f5c1e726a24ae87af9b" containerID="232650c1f395159ab665c2ac60172410d0888b5c779860632cb98f89b7318aff" exitCode=0
Jul 09 18:25:27.390088 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.390001    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal" event={"ID":"14f0bfd5e98f4f5c1e726a24ae87af9b","Type":"ContainerDied","Data":"232650c1f395159ab665c2ac60172410d0888b5c779860632cb98f89b7318aff"}
Jul 09 18:25:27.470871 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:27.470812    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-28.ec2.internal" podStartSLOduration=3.470792338 podStartE2EDuration="3.470792338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:25:27.428109592 +0000 UTC m=+4.563927068" watchObservedRunningTime="2026-07-09 18:25:27.470792338 +0000 UTC m=+4.606609814"
Jul 09 18:25:28.406368 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:28.406298    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal" event={"ID":"14f0bfd5e98f4f5c1e726a24ae87af9b","Type":"ContainerStarted","Data":"ae03a14758e33d29f270319dfd834e1bd83c146b28988c8fefabfc418303bd95"}
Jul 09 18:25:28.837431 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:28.836685    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:28.837431 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:28.836740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:28.837431 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:28.836896    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:28.837431 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:28.836954    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret podName:9876487f-ee1d-4875-940e-d0032bc17427 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:32.836936889 +0000 UTC m=+9.972754342 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret") pod "global-pull-secret-syncer-7m46n" (UID: "9876487f-ee1d-4875-940e-d0032bc17427") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:28.837431 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:28.837018    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:28.837431 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:28.837048    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs podName:735bceb1-4f3b-4163-a164-a70f462b5aac nodeName:}" failed. No retries permitted until 2026-07-09 18:25:32.837037679 +0000 UTC m=+9.972855134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs") pod "network-metrics-daemon-llr8j" (UID: "735bceb1-4f3b-4163-a164-a70f462b5aac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:29.038958 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:29.038297    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qkml\" (UniqueName: \"kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml\") pod \"network-check-target-xx64v\" (UID: \"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0\") " pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:29.038958 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:29.038502    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:25:29.038958 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:29.038526    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:25:29.038958 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:29.038538    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2qkml for pod openshift-network-diagnostics/network-check-target-xx64v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:29.038958 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:29.038591    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml podName:962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:33.038573722 +0000 UTC m=+10.174391175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qkml" (UniqueName: "kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml") pod "network-check-target-xx64v" (UID: "962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:29.353671 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:29.353617    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:29.353832 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:29.353754    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:29.354166 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:29.354145    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:29.354259 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:29.354236    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:29.354529 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:29.354399    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:29.354529 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:29.354497    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:31.355573 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:31.355541    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:31.356026 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:31.355693    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:31.356085 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:31.356055    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:31.356085 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:31.356068    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:31.356188 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:31.356149    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:31.356299 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:31.356237    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:32.872160 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:32.871758    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:32.872160 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:32.871806    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:32.872160 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:32.871926    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:32.872160 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:32.871983    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret podName:9876487f-ee1d-4875-940e-d0032bc17427 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:40.87196438 +0000 UTC m=+18.007781847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret") pod "global-pull-secret-syncer-7m46n" (UID: "9876487f-ee1d-4875-940e-d0032bc17427") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:32.872160 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:32.872070    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:32.872160 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:32.872119    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs podName:735bceb1-4f3b-4163-a164-a70f462b5aac nodeName:}" failed. No retries permitted until 2026-07-09 18:25:40.872106328 +0000 UTC m=+18.007923785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs") pod "network-metrics-daemon-llr8j" (UID: "735bceb1-4f3b-4163-a164-a70f462b5aac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:33.073890 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:33.073827    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qkml\" (UniqueName: \"kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml\") pod \"network-check-target-xx64v\" (UID: \"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0\") " pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:33.074073 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:33.074018    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:25:33.074073 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:33.074045    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:25:33.074073 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:33.074059    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2qkml for pod openshift-network-diagnostics/network-check-target-xx64v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:33.074228 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:33.074122    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml podName:962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:41.074103206 +0000 UTC m=+18.209920659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qkml" (UniqueName: "kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml") pod "network-check-target-xx64v" (UID: "962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:33.354451 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:33.354422    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:33.354619 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:33.354526    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:33.354703 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:33.354617    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:33.354755 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:33.354735    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:33.354809 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:33.354782    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:33.354892 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:33.354865    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:35.353455 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:35.353421    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:35.353455 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:35.353447    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:35.353869 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:35.353423    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:35.353869 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:35.353519    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:35.353869 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:35.353602    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:35.353869 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:35.353680    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:36.425417 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.425165    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v2wcw" event={"ID":"67607ff1-affa-4647-965e-119f2a24637f","Type":"ContainerStarted","Data":"fe0eaa8e76c2147c633ce486e1f68812f51a56631c0662f4de56f613339e3b46"}
Jul 09 18:25:36.427080 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.427049    2566 generic.go:356] "Generic (PLEG): container finished" podID="64f7b4ed-439b-408d-9b3b-cb169308ab35" containerID="54dafa4a790cccdbabc90d9be4c6dc09508272e73ce45ea51e08784ba9dc2728" exitCode=0
Jul 09 18:25:36.427200 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.427128    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zms9" event={"ID":"64f7b4ed-439b-408d-9b3b-cb169308ab35","Type":"ContainerDied","Data":"54dafa4a790cccdbabc90d9be4c6dc09508272e73ce45ea51e08784ba9dc2728"}
Jul 09 18:25:36.429219 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.429171    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-slptj" event={"ID":"07a65c98-48c0-4cd8-a737-5908696e29b6","Type":"ContainerStarted","Data":"c297f31063400bcc114f9fb16e9820e85d4a7234e114c4bbae9cb7cb8228183b"}
Jul 09 18:25:36.430920 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.430895    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2mb6q" event={"ID":"59b49275-d607-420f-acaf-10d3b9843b44","Type":"ContainerStarted","Data":"81e0d27cca4ba9e3d7bccfeb4b0b8048af81c011f562b725fea2b504c0e1413c"}
Jul 09 18:25:36.433033 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.432998    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9wx8q" event={"ID":"2bb32097-7550-49c6-86fa-5d8248ace642","Type":"ContainerStarted","Data":"c3ebb4648f9cb6ae4aa3425bb3de3c660c2c8ebe8f2fc6cbb06460f1e6c8fb61"}
Jul 09 18:25:36.434554 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.434534    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn" event={"ID":"90c26f7b-938e-4b61-972c-8e1ff90c4b08","Type":"ContainerStarted","Data":"bcc39514ab8fe7889ccba74c02db8c5c3181b736666c54a2dbbb17524d927183"}
Jul 09 18:25:36.449886 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.449843    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-28.ec2.internal" podStartSLOduration=12.449828292 podStartE2EDuration="12.449828292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:25:28.426333413 +0000 UTC m=+5.562150887" watchObservedRunningTime="2026-07-09 18:25:36.449828292 +0000 UTC m=+13.585645769"
Jul 09 18:25:36.499163 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.499116    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v2wcw" podStartSLOduration=3.20940714 podStartE2EDuration="13.499101266s" totalImagesPullingTime="10.289694126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:25:36.449918624 +0000 UTC m=+13.585736098" watchObservedRunningTime="2026-07-09 18:25:36.499101266 +0000 UTC m=+13.634918741"
Jul 09 18:25:36.499306 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.499234    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2mb6q" podStartSLOduration=3.177034861 podStartE2EDuration="13.499226415s" totalImagesPullingTime="10.322191554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:25:36.498967303 +0000 UTC m=+13.634784775" watchObservedRunningTime="2026-07-09 18:25:36.499226415 +0000 UTC m=+13.635043889"
Jul 09 18:25:36.609611 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.609561    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-slptj" podStartSLOduration=3.319515301 podStartE2EDuration="13.609542183s" totalImagesPullingTime="10.290026882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:25:36.552667693 +0000 UTC m=+13.688485161" watchObservedRunningTime="2026-07-09 18:25:36.609542183 +0000 UTC m=+13.745359656"
Jul 09 18:25:36.667840 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:36.666762    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9wx8q" podStartSLOduration=3.426718198 podStartE2EDuration="13.666741047s" totalImagesPullingTime="10.240022849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:25:36.613177457 +0000 UTC m=+13.748994931" watchObservedRunningTime="2026-07-09 18:25:36.666741047 +0000 UTC m=+13.802558518"
Jul 09 18:25:37.353501 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:37.353466    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:37.353665 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:37.353564    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:37.353732 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:37.353683    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:37.353794 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:37.353776    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:37.353834 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:37.353806    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:37.353883 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:37.353856    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:37.437741 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:37.437686    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rjbtr" event={"ID":"f98f059e-caca-400d-b05c-5d18371deac1","Type":"ContainerStarted","Data":"8f39c36e470dc0c307d3a657582ad159e047939a24f2d80f28b0928d2cba827f"}
Jul 09 18:25:37.483292 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:37.483234    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rjbtr" podStartSLOduration=4.246927872 podStartE2EDuration="14.483215245s" totalImagesPullingTime="10.236287373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:25:37.480131051 +0000 UTC m=+14.615948522" watchObservedRunningTime="2026-07-09 18:25:37.483215245 +0000 UTC m=+14.619032725"
Jul 09 18:25:37.640640 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:37.640612    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 18:25:38.280267 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:38.280171    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T18:25:37.640636625Z","UUID":"ea1e0059-fb57-4152-9d24-743b24f8fd39","Handler":null,"Name":"","Endpoint":""}
Jul 09 18:25:38.281719 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:38.281696    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 18:25:38.281843 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:38.281743    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 18:25:38.441421 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:38.441379    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn" event={"ID":"90c26f7b-938e-4b61-972c-8e1ff90c4b08","Type":"ContainerStarted","Data":"a45738bad787dd6aa2624b5fc7c6fa07e8de9a6c3e57b3e7b3245cbeb6d6ab3c"}
Jul 09 18:25:38.485074 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:38.485041    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:38.485734 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:38.485715    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:39.353466 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:39.353287    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:39.353623 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:39.353287    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:39.353623 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:39.353571    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:39.353765 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:39.353301    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:39.353765 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:39.353658    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:39.353765 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:39.353749    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:39.445496 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:39.445376    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn" event={"ID":"90c26f7b-938e-4b61-972c-8e1ff90c4b08","Type":"ContainerStarted","Data":"80d9c8f93b48e2e83ec5d137587b75e9bdc5df251635fa22e05d8533a07da034"}
Jul 09 18:25:39.445932 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:39.445549    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:39.446047 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:39.446027    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9wx8q"
Jul 09 18:25:39.470901 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:39.470847    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbdhn" podStartSLOduration=3.289598627 podStartE2EDuration="16.470830181s" totalImagesPullingTime="13.181231554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:25:39.46732203 +0000 UTC m=+16.603139503" watchObservedRunningTime="2026-07-09 18:25:39.470830181 +0000 UTC m=+16.606647654"
Jul 09 18:25:40.932710 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:40.932674    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:40.933324 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:40.932727    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:40.933324 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:40.932839    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:40.933324 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:40.932841    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:40.933324 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:40.932899    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret podName:9876487f-ee1d-4875-940e-d0032bc17427 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:56.93288137 +0000 UTC m=+34.068698836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret") pod "global-pull-secret-syncer-7m46n" (UID: "9876487f-ee1d-4875-940e-d0032bc17427") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:40.933324 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:40.932920    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs podName:735bceb1-4f3b-4163-a164-a70f462b5aac nodeName:}" failed. No retries permitted until 2026-07-09 18:25:56.932909418 +0000 UTC m=+34.068726871 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs") pod "network-metrics-daemon-llr8j" (UID: "735bceb1-4f3b-4163-a164-a70f462b5aac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:41.134434 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:41.134394    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qkml\" (UniqueName: \"kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml\") pod \"network-check-target-xx64v\" (UID: \"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0\") " pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:41.134591 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:41.134571    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:25:41.134696 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:41.134593    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:25:41.134696 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:41.134603    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2qkml for pod openshift-network-diagnostics/network-check-target-xx64v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:41.134696 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:41.134667    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml podName:962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0 nodeName:}" failed. No retries permitted until 2026-07-09 18:25:57.134636368 +0000 UTC m=+34.270453818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qkml" (UniqueName: "kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml") pod "network-check-target-xx64v" (UID: "962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:41.354451 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:41.354373    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:41.354608 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:41.354505    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:41.354915 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:41.354892    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:41.355030 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:41.354990    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:41.355175 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:41.355151    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:41.355308 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:41.355275    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:43.355032 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:43.354997    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:43.355032 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:43.355029    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:43.355557 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:43.355110    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:43.355557 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:43.355426    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:43.355557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:43.355464    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:43.355557 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:43.355523    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:45.354245 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:45.354214    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:45.354806 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:45.354256    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:45.354806 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:45.354219    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:45.354806 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:45.354349    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:45.354806 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:45.354438    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:45.354806 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:45.354523    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:47.353236 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:47.353200    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:47.353781 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:47.353201    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:47.353781 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:47.353336    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:47.353781 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:47.353201    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:47.353781 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:47.353410    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:47.353781 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:47.353477    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:49.354197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:49.354160    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:49.354632 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:49.354161    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:49.354632 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:49.354292    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:49.354632 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:49.354168    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:49.354632 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:49.354376    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:49.354632 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:49.354429    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:51.354079 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.353925    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:51.354664 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.353931    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:51.354664 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:51.354150    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:51.354664 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.353958    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:51.354664 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:51.354227    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:51.354664 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:51.354336    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:51.468208 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.468180    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rmknh" event={"ID":"0fbe583d-3a9d-419b-867d-4a0c1f4f0580","Type":"ContainerStarted","Data":"3b3290a82c5a3b6f5a9e18abacb508de0ebe9a33255cd2a529cd77312bbcd21f"}
Jul 09 18:25:51.469970 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.469946    2566 generic.go:356] "Generic (PLEG): container finished" podID="64f7b4ed-439b-408d-9b3b-cb169308ab35" containerID="97a34975c95ada7c5c318bff5300243e20c0b30c47c2ac32515d9c96c227be3c" exitCode=0
Jul 09 18:25:51.470055 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.470029    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zms9" event={"ID":"64f7b4ed-439b-408d-9b3b-cb169308ab35","Type":"ContainerDied","Data":"97a34975c95ada7c5c318bff5300243e20c0b30c47c2ac32515d9c96c227be3c"}
Jul 09 18:25:51.472547 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.472527    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq" event={"ID":"ab113693-b417-46df-895a-91f05883b294","Type":"ContainerStarted","Data":"b0347c5c7d58c0437af953830052000cee93651f4d02cfd73260ef40f617a3fc"}
Jul 09 18:25:51.472635 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.472586    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq" event={"ID":"ab113693-b417-46df-895a-91f05883b294","Type":"ContainerStarted","Data":"55b77c1bcf95255d79ed9d4ac47b926d9a4059ce119f4d5b50d6d088f33be284"}
Jul 09 18:25:51.472635 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.472618    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq" event={"ID":"ab113693-b417-46df-895a-91f05883b294","Type":"ContainerStarted","Data":"d1a31340dd81f1eaa0189e480afeeceecf11c3f49870743f8d576ae2e4e6efbf"}
Jul 09 18:25:51.472729 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.472666    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq" event={"ID":"ab113693-b417-46df-895a-91f05883b294","Type":"ContainerStarted","Data":"8b879a9a91f45e4c6b7c39734298830f35c66cd52a9f5bb6795a38d0259ee76c"}
Jul 09 18:25:51.472729 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.472696    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq" event={"ID":"ab113693-b417-46df-895a-91f05883b294","Type":"ContainerStarted","Data":"ebc5d4017a005821ab6f43d5cc5c59ad8b52bb5ff52c33721282c5e319a0e328"}
Jul 09 18:25:51.498000 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:51.497968    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rmknh" podStartSLOduration=3.210661179 podStartE2EDuration="28.497956436s" totalImagesPullingTime="25.287295257s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:25:51.497309699 +0000 UTC m=+28.633127171" watchObservedRunningTime="2026-07-09 18:25:51.497956436 +0000 UTC m=+28.633773908"
Jul 09 18:25:52.478339 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:52.478286    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq" event={"ID":"ab113693-b417-46df-895a-91f05883b294","Type":"ContainerStarted","Data":"dfdf9c94b34163c657e878856b46bd4e01db54e0382fcc8e636e0d7b9e9e45d8"}
Jul 09 18:25:53.354076 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:53.354050    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:53.354226 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:53.354138    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:53.354226 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:53.354186    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:53.354308 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:53.354246    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:53.354308 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:53.354294    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:53.354388 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:53.354372    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:53.482493 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:53.482461    2566 generic.go:356] "Generic (PLEG): container finished" podID="64f7b4ed-439b-408d-9b3b-cb169308ab35" containerID="3327d0fc3398d78dedaa74e2ecfa55f3da247d3494ec5f275f8d2a6f84a0afc5" exitCode=0
Jul 09 18:25:53.483037 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:53.482509    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zms9" event={"ID":"64f7b4ed-439b-408d-9b3b-cb169308ab35","Type":"ContainerDied","Data":"3327d0fc3398d78dedaa74e2ecfa55f3da247d3494ec5f275f8d2a6f84a0afc5"}
Jul 09 18:25:54.488856 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:54.488823    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq" event={"ID":"ab113693-b417-46df-895a-91f05883b294","Type":"ContainerStarted","Data":"c64a6ba0f8d0583dbcc8f6184985a8506e94dbca51cae84383803aa1a24947f1"}
Jul 09 18:25:55.353793 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:55.353757    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:55.353960 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:55.353762    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:55.353960 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:55.353857    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:55.353960 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:55.353762    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:55.354122 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:55.353942    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:55.354122 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:55.354028    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:55.493190 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:55.493157    2566 generic.go:356] "Generic (PLEG): container finished" podID="64f7b4ed-439b-408d-9b3b-cb169308ab35" containerID="7bd1060e996c5efb2e1714412775189ae49ff0defffba0c5a8a6356aa69e9849" exitCode=0
Jul 09 18:25:55.493613 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:55.493220    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zms9" event={"ID":"64f7b4ed-439b-408d-9b3b-cb169308ab35","Type":"ContainerDied","Data":"7bd1060e996c5efb2e1714412775189ae49ff0defffba0c5a8a6356aa69e9849"}
Jul 09 18:25:56.498921 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:56.498740    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq" event={"ID":"ab113693-b417-46df-895a-91f05883b294","Type":"ContainerStarted","Data":"5b27396d4d645f7d338a5b83187581f184b5731db510a4144b84fe40b4be035b"}
Jul 09 18:25:56.499311 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:56.499003    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:56.499311 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:56.499042    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:56.514168 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:56.514146    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:56.531119 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:56.531078    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq" podStartSLOduration=8.210704384 podStartE2EDuration="33.531064745s" totalImagesPullingTime="25.320360361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:25:56.530698441 +0000 UTC m=+33.666515914" watchObservedRunningTime="2026-07-09 18:25:56.531064745 +0000 UTC m=+33.666882217"
Jul 09 18:25:56.953331 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:56.953301    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:56.953470 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:56.953343    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:56.953470 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:56.953436    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:56.953470 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:56.953441    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:56.953567 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:56.953481    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret podName:9876487f-ee1d-4875-940e-d0032bc17427 nodeName:}" failed. No retries permitted until 2026-07-09 18:26:28.953469564 +0000 UTC m=+66.089287016 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret") pod "global-pull-secret-syncer-7m46n" (UID: "9876487f-ee1d-4875-940e-d0032bc17427") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:25:56.953567 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:56.953495    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs podName:735bceb1-4f3b-4163-a164-a70f462b5aac nodeName:}" failed. No retries permitted until 2026-07-09 18:26:28.953488255 +0000 UTC m=+66.089305705 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs") pod "network-metrics-daemon-llr8j" (UID: "735bceb1-4f3b-4163-a164-a70f462b5aac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:25:57.154544 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:57.154505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qkml\" (UniqueName: \"kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml\") pod \"network-check-target-xx64v\" (UID: \"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0\") " pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:57.154742 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:57.154694    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:25:57.154742 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:57.154722    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:25:57.154742 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:57.154736    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2qkml for pod openshift-network-diagnostics/network-check-target-xx64v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:57.154868 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:57.154791    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml podName:962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0 nodeName:}" failed. No retries permitted until 2026-07-09 18:26:29.154772522 +0000 UTC m=+66.290589977 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qkml" (UniqueName: "kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml") pod "network-check-target-xx64v" (UID: "962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:25:57.354258 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:57.354176    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:57.354417 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:57.354176    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:57.354417 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:57.354288    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:57.354417 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:57.354288    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:57.354417 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:57.354349    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:57.354552 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:57.354412    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:57.501240 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:57.501211    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:57.518794 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:57.518769    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:25:58.177029 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:58.176997    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-llr8j"]
Jul 09 18:25:58.177172 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:58.177094    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:25:58.177217 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:58.177190    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:25:58.181102 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:58.181071    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xx64v"]
Jul 09 18:25:58.181240 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:58.181214    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:25:58.181345 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:58.181324    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:25:58.183767 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:58.183741    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7m46n"]
Jul 09 18:25:58.183879 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:58.183826    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:58.183946 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:58.183897    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:25:59.353809 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:25:59.353777    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:25:59.354243 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:25:59.353907    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:26:00.353792 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:00.353758    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:26:00.353792 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:00.353786    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:26:00.354265 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:00.353884    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:26:00.354265 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:00.353995    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:26:01.353522 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:01.353489    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:26:01.353700 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:01.353625    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:26:02.353792 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:02.353609    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:26:02.354232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:02.353694    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:26:02.354232 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:02.353859    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx64v" podUID="962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0"
Jul 09 18:26:02.354232 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:02.353953    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-llr8j" podUID="735bceb1-4f3b-4163-a164-a70f462b5aac"
Jul 09 18:26:02.513738 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:02.513704    2566 generic.go:356] "Generic (PLEG): container finished" podID="64f7b4ed-439b-408d-9b3b-cb169308ab35" containerID="5a1654cc77e79bd0d9fd1a8e8540836819660fcb53ee365d28bb46b8b9f973f1" exitCode=0
Jul 09 18:26:02.513867 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:02.513758    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zms9" event={"ID":"64f7b4ed-439b-408d-9b3b-cb169308ab35","Type":"ContainerDied","Data":"5a1654cc77e79bd0d9fd1a8e8540836819660fcb53ee365d28bb46b8b9f973f1"}
Jul 09 18:26:03.354211 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.354180    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:26:03.354850 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:03.354253    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7m46n" podUID="9876487f-ee1d-4875-940e-d0032bc17427"
Jul 09 18:26:03.518307 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.518230    2566 generic.go:356] "Generic (PLEG): container finished" podID="64f7b4ed-439b-408d-9b3b-cb169308ab35" containerID="510a93354190a160ae285ee8ceb59038dc7ddf31827b220555fa39e1675ef6f9" exitCode=0
Jul 09 18:26:03.518307 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.518293    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zms9" event={"ID":"64f7b4ed-439b-408d-9b3b-cb169308ab35","Type":"ContainerDied","Data":"510a93354190a160ae285ee8ceb59038dc7ddf31827b220555fa39e1675ef6f9"}
Jul 09 18:26:03.696675 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.696637    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-28.ec2.internal" event="NodeReady"
Jul 09 18:26:03.696794 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.696783    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 18:26:03.747762 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.747736    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bdc4656cf-frhhm"]
Jul 09 18:26:03.757462 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.757442    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.760237 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.760218    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 18:26:03.779616 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.779554    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lxd8z\""
Jul 09 18:26:03.787425 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.787408    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 18:26:03.787515 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.787505    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 18:26:03.800186 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.800167    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bdc4656cf-frhhm"]
Jul 09 18:26:03.803708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.803496    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32c563e6-5352-453d-b256-e3ed5819a18c-ca-trust-extracted\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.803708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.803534    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-trusted-ca\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.803708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.803559    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltsdz\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-kube-api-access-ltsdz\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.803708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.803628    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.803708 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.803690    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-installation-pull-secrets\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.804004 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.803715    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-registry-certificates\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.804004 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.803766    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-image-registry-private-configuration\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.804004 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.803790    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-bound-sa-token\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.804374 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.804359    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xdlwn"]
Jul 09 18:26:03.820410 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.820389    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 18:26:03.820634 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.820619    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xv799"]
Jul 09 18:26:03.820759 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.820746    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xdlwn"
Jul 09 18:26:03.826284 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.826268    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 18:26:03.826585 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.826566    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 18:26:03.827164 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.827152    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 18:26:03.827354 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.827339    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dqpcm\""
Jul 09 18:26:03.837040 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.837018    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xv799"]
Jul 09 18:26:03.837147 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.837108    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:03.840105 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.840087    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 18:26:03.840756 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.840743    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 18:26:03.841949 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.841933    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jnv68\""
Jul 09 18:26:03.851190 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.851171    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xdlwn"]
Jul 09 18:26:03.904882 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.904860    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.904999 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.904899    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-installation-pull-secrets\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.904999 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.904918    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-registry-certificates\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.904999 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.904938    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/274a4b22-0f83-4378-a747-f6a741e5abf3-cert\") pod \"ingress-canary-xdlwn\" (UID: \"274a4b22-0f83-4378-a747-f6a741e5abf3\") " pod="openshift-ingress-canary/ingress-canary-xdlwn"
Jul 09 18:26:03.905159 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905019    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-image-registry-private-configuration\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.905159 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:03.905028    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:26:03.905159 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:03.905046    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bdc4656cf-frhhm: secret "image-registry-tls" not found
Jul 09 18:26:03.905159 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:03.905103    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls podName:32c563e6-5352-453d-b256-e3ed5819a18c nodeName:}" failed. No retries permitted until 2026-07-09 18:26:04.405082187 +0000 UTC m=+41.540899642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls") pod "image-registry-5bdc4656cf-frhhm" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c") : secret "image-registry-tls" not found
Jul 09 18:26:03.905159 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905043    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-bound-sa-token\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.905159 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905154    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f37ae45-4062-40db-9fc7-6fcec23b742e-tmp-dir\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:03.905471 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905189    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32c563e6-5352-453d-b256-e3ed5819a18c-ca-trust-extracted\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.905471 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905220    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-trusted-ca\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.905471 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905243    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltsdz\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-kube-api-access-ltsdz\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.905471 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905268    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mjsl\" (UniqueName: \"kubernetes.io/projected/6f37ae45-4062-40db-9fc7-6fcec23b742e-kube-api-access-7mjsl\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:03.905471 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905296    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njldq\" (UniqueName: \"kubernetes.io/projected/274a4b22-0f83-4378-a747-f6a741e5abf3-kube-api-access-njldq\") pod \"ingress-canary-xdlwn\" (UID: \"274a4b22-0f83-4378-a747-f6a741e5abf3\") " pod="openshift-ingress-canary/ingress-canary-xdlwn"
Jul 09 18:26:03.905471 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905319    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6f37ae45-4062-40db-9fc7-6fcec23b742e-config-volume\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:03.905471 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905338    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:03.905760 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905487    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-registry-certificates\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.905760 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.905545    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32c563e6-5352-453d-b256-e3ed5819a18c-ca-trust-extracted\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.906173 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.906150    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-trusted-ca\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.908837 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.908819    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-installation-pull-secrets\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.908936 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.908835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-image-registry-private-configuration\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.926722 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.926688    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltsdz\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-kube-api-access-ltsdz\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:03.927339 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:03.927324    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-bound-sa-token\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:04.006363 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.006339    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f37ae45-4062-40db-9fc7-6fcec23b742e-tmp-dir\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:04.006363 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.006372    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mjsl\" (UniqueName: \"kubernetes.io/projected/6f37ae45-4062-40db-9fc7-6fcec23b742e-kube-api-access-7mjsl\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:04.006580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.006390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njldq\" (UniqueName: \"kubernetes.io/projected/274a4b22-0f83-4378-a747-f6a741e5abf3-kube-api-access-njldq\") pod \"ingress-canary-xdlwn\" (UID: \"274a4b22-0f83-4378-a747-f6a741e5abf3\") " pod="openshift-ingress-canary/ingress-canary-xdlwn"
Jul 09 18:26:04.006580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.006407    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6f37ae45-4062-40db-9fc7-6fcec23b742e-config-volume\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:04.006580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.006420    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:04.006580 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:04.006496    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:26:04.006580 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:04.006538    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls podName:6f37ae45-4062-40db-9fc7-6fcec23b742e nodeName:}" failed. No retries permitted until 2026-07-09 18:26:04.506523165 +0000 UTC m=+41.642340615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls") pod "dns-default-xv799" (UID: "6f37ae45-4062-40db-9fc7-6fcec23b742e") : secret "dns-default-metrics-tls" not found
Jul 09 18:26:04.006867 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.006637    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/274a4b22-0f83-4378-a747-f6a741e5abf3-cert\") pod \"ingress-canary-xdlwn\" (UID: \"274a4b22-0f83-4378-a747-f6a741e5abf3\") " pod="openshift-ingress-canary/ingress-canary-xdlwn"
Jul 09 18:26:04.006867 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.006698    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f37ae45-4062-40db-9fc7-6fcec23b742e-tmp-dir\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:04.007018 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.006999    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6f37ae45-4062-40db-9fc7-6fcec23b742e-config-volume\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:04.008614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.008597    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/274a4b22-0f83-4378-a747-f6a741e5abf3-cert\") pod \"ingress-canary-xdlwn\" (UID: \"274a4b22-0f83-4378-a747-f6a741e5abf3\") " pod="openshift-ingress-canary/ingress-canary-xdlwn"
Jul 09 18:26:04.023550 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.023529    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njldq\" (UniqueName: \"kubernetes.io/projected/274a4b22-0f83-4378-a747-f6a741e5abf3-kube-api-access-njldq\") pod \"ingress-canary-xdlwn\" (UID: \"274a4b22-0f83-4378-a747-f6a741e5abf3\") " pod="openshift-ingress-canary/ingress-canary-xdlwn"
Jul 09 18:26:04.024676 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.024635    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mjsl\" (UniqueName: \"kubernetes.io/projected/6f37ae45-4062-40db-9fc7-6fcec23b742e-kube-api-access-7mjsl\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:04.128624 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.128575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xdlwn"
Jul 09 18:26:04.325375 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.325203    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xdlwn"]
Jul 09 18:26:04.328772 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:04.328746    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274a4b22_0f83_4378_a747_f6a741e5abf3.slice/crio-90973ac897507127d831c39d38e767855549d28a106c31f1bbd7fd7e051b0d61 WatchSource:0}: Error finding container 90973ac897507127d831c39d38e767855549d28a106c31f1bbd7fd7e051b0d61: Status 404 returned error can't find the container with id 90973ac897507127d831c39d38e767855549d28a106c31f1bbd7fd7e051b0d61
Jul 09 18:26:04.354320 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.354112    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:26:04.354813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.354367    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:26:04.357972 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.357949    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 18:26:04.360805 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.360787    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 18:26:04.360936 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.360915    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 18:26:04.361076 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.361048    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7pc9s\""
Jul 09 18:26:04.361129 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.361098    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qx56j\""
Jul 09 18:26:04.410351 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.410266    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:04.410448 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:04.410403    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:26:04.410448 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:04.410420    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bdc4656cf-frhhm: secret "image-registry-tls" not found
Jul 09 18:26:04.410538 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:04.410478    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls podName:32c563e6-5352-453d-b256-e3ed5819a18c nodeName:}" failed. No retries permitted until 2026-07-09 18:26:05.410459457 +0000 UTC m=+42.546276925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls") pod "image-registry-5bdc4656cf-frhhm" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c") : secret "image-registry-tls" not found
Jul 09 18:26:04.511477 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.511454    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:04.511577 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:04.511546    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:26:04.511623 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:04.511594    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls podName:6f37ae45-4062-40db-9fc7-6fcec23b742e nodeName:}" failed. No retries permitted until 2026-07-09 18:26:05.511581236 +0000 UTC m=+42.647398685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls") pod "dns-default-xv799" (UID: "6f37ae45-4062-40db-9fc7-6fcec23b742e") : secret "dns-default-metrics-tls" not found
Jul 09 18:26:04.521215 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.521188    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xdlwn" event={"ID":"274a4b22-0f83-4378-a747-f6a741e5abf3","Type":"ContainerStarted","Data":"90973ac897507127d831c39d38e767855549d28a106c31f1bbd7fd7e051b0d61"}
Jul 09 18:26:04.524389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.524368    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zms9" event={"ID":"64f7b4ed-439b-408d-9b3b-cb169308ab35","Type":"ContainerStarted","Data":"27f017e73c2b43b0a355ea9f546f7ac6f945c44003c41539518f815b4ae2e778"}
Jul 09 18:26:04.549622 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:04.549584    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6zms9" podStartSLOduration=7.628517739 podStartE2EDuration="41.549572304s" totalImagesPullingTime="33.921054565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:04.548504311 +0000 UTC m=+41.684321796" watchObservedRunningTime="2026-07-09 18:26:04.549572304 +0000 UTC m=+41.685389776"
Jul 09 18:26:05.353707 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:05.353676    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:26:05.356482 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:05.356461    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 18:26:05.419364 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:05.419331    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:05.419536 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:05.419516    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:26:05.419625 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:05.419538    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bdc4656cf-frhhm: secret "image-registry-tls" not found
Jul 09 18:26:05.419625 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:05.419597    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls podName:32c563e6-5352-453d-b256-e3ed5819a18c nodeName:}" failed. No retries permitted until 2026-07-09 18:26:07.419580186 +0000 UTC m=+44.555397641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls") pod "image-registry-5bdc4656cf-frhhm" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c") : secret "image-registry-tls" not found
Jul 09 18:26:05.520168 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:05.520128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:05.520293 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:05.520267    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:26:05.520356 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:05.520345    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls podName:6f37ae45-4062-40db-9fc7-6fcec23b742e nodeName:}" failed. No retries permitted until 2026-07-09 18:26:07.520326209 +0000 UTC m=+44.656143672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls") pod "dns-default-xv799" (UID: "6f37ae45-4062-40db-9fc7-6fcec23b742e") : secret "dns-default-metrics-tls" not found
Jul 09 18:26:07.436384 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:07.436350    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:07.436841 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:07.436518    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:26:07.436841 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:07.436539    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bdc4656cf-frhhm: secret "image-registry-tls" not found
Jul 09 18:26:07.436841 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:07.436604    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls podName:32c563e6-5352-453d-b256-e3ed5819a18c nodeName:}" failed. No retries permitted until 2026-07-09 18:26:11.436585206 +0000 UTC m=+48.572402656 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls") pod "image-registry-5bdc4656cf-frhhm" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c") : secret "image-registry-tls" not found
Jul 09 18:26:07.537065 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:07.537031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:07.537227 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:07.537151    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:26:07.537227 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:07.537220    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls podName:6f37ae45-4062-40db-9fc7-6fcec23b742e nodeName:}" failed. No retries permitted until 2026-07-09 18:26:11.537202982 +0000 UTC m=+48.673020434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls") pod "dns-default-xv799" (UID: "6f37ae45-4062-40db-9fc7-6fcec23b742e") : secret "dns-default-metrics-tls" not found
Jul 09 18:26:08.536133 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:08.535892    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xdlwn" event={"ID":"274a4b22-0f83-4378-a747-f6a741e5abf3","Type":"ContainerStarted","Data":"47ae5c4ddd5f72505327d482be2b2357bde25f85d299bd159c8faa19b8b63d23"}
Jul 09 18:26:08.556140 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:08.556076    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xdlwn" podStartSLOduration=2.221911319 podStartE2EDuration="5.556059403s" totalImagesPullingTime="3.334148084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:08.555506444 +0000 UTC m=+45.691323916" watchObservedRunningTime="2026-07-09 18:26:08.556059403 +0000 UTC m=+45.691876853"
Jul 09 18:26:11.464577 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:11.464535    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:11.465086 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:11.464704    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:26:11.465086 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:11.464722    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bdc4656cf-frhhm: secret "image-registry-tls" not found
Jul 09 18:26:11.465086 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:11.464775    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls podName:32c563e6-5352-453d-b256-e3ed5819a18c nodeName:}" failed. No retries permitted until 2026-07-09 18:26:19.464760535 +0000 UTC m=+56.600577989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls") pod "image-registry-5bdc4656cf-frhhm" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c") : secret "image-registry-tls" not found
Jul 09 18:26:11.565386 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:11.565356    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:11.565536 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:11.565473    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:26:11.565536 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:11.565522    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls podName:6f37ae45-4062-40db-9fc7-6fcec23b742e nodeName:}" failed. No retries permitted until 2026-07-09 18:26:19.565508208 +0000 UTC m=+56.701325657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls") pod "dns-default-xv799" (UID: "6f37ae45-4062-40db-9fc7-6fcec23b742e") : secret "dns-default-metrics-tls" not found
Jul 09 18:26:12.754555 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.754518    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"]
Jul 09 18:26:12.776532 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.776502    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"]
Jul 09 18:26:12.776694 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.776595    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:12.783211 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.783185    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 09 18:26:12.783362 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.783184    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4c5k9\""
Jul 09 18:26:12.784046 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.784030    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 09 18:26:12.784293 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.784275    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:26:12.864666 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.864620    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-ctcrt"]
Jul 09 18:26:12.874563 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.874532    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:12.874701 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.874568    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm7w8\" (UniqueName: \"kubernetes.io/projected/bc82e223-fb55-4ae1-8b06-011f014427f1-kube-api-access-qm7w8\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:12.878327 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.878308    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"]
Jul 09 18:26:12.878437 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.878401    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:12.888513 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.888492    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb"]
Jul 09 18:26:12.888634 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.888617    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:12.890278 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.890257    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 09 18:26:12.890557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.890539    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kldpc\""
Jul 09 18:26:12.890727 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.890599    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 09 18:26:12.890816 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.890668    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 18:26:12.894982 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.894954    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 18:26:12.895093 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.895001    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 18:26:12.895093 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.895024    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 18:26:12.896108 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.896094    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nb5gr\""
Jul 09 18:26:12.896168 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.896156    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 09 18:26:12.896832 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.896817    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 09 18:26:12.903365 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.903342    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 09 18:26:12.904432 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.904413    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-ctcrt"]
Jul 09 18:26:12.904497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.904442    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb"]
Jul 09 18:26:12.904535 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.904513    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb"
Jul 09 18:26:12.918104 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.918084    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jn9jv\""
Jul 09 18:26:12.934983 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.934957    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"]
Jul 09 18:26:12.975507 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975469    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7722b6-b03d-4160-89d5-a7aaad6d122e-tmp\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:12.975507 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975502    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ee7722b6-b03d-4160-89d5-a7aaad6d122e-snapshots\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:12.975819 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975519    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6cv7\" (UniqueName: \"kubernetes.io/projected/ee7722b6-b03d-4160-89d5-a7aaad6d122e-kube-api-access-l6cv7\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:12.975819 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975537    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7722b6-b03d-4160-89d5-a7aaad6d122e-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:12.975819 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975554    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7722b6-b03d-4160-89d5-a7aaad6d122e-serving-cert\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:12.975819 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:12.975819 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975638    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:12.975819 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975673    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/87da33a9-c88f-4f09-ac99-9bc91c35d81f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:12.975819 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975697    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm7w8\" (UniqueName: \"kubernetes.io/projected/bc82e223-fb55-4ae1-8b06-011f014427f1-kube-api-access-qm7w8\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:12.975819 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975712    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7722b6-b03d-4160-89d5-a7aaad6d122e-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:12.975819 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:12.975750    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:26:12.975819 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975784    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79ldn\" (UniqueName: \"kubernetes.io/projected/87da33a9-c88f-4f09-ac99-9bc91c35d81f-kube-api-access-79ldn\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:12.976226 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.975867    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z58q\" (UniqueName: \"kubernetes.io/projected/1079be64-5c3f-4df5-9437-034a2eab9c8e-kube-api-access-9z58q\") pod \"network-check-source-869dcfcdcd-6c5jb\" (UID: \"1079be64-5c3f-4df5-9437-034a2eab9c8e\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb"
Jul 09 18:26:12.976226 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:12.975903    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls podName:bc82e223-fb55-4ae1-8b06-011f014427f1 nodeName:}" failed. No retries permitted until 2026-07-09 18:26:13.475879558 +0000 UTC m=+50.611697021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls") pod "cluster-samples-operator-65db858894-gfmvf" (UID: "bc82e223-fb55-4ae1-8b06-011f014427f1") : secret "samples-operator-tls" not found
Jul 09 18:26:12.993015 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:12.992982    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm7w8\" (UniqueName: \"kubernetes.io/projected/bc82e223-fb55-4ae1-8b06-011f014427f1-kube-api-access-qm7w8\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:13.076292 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076214    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7722b6-b03d-4160-89d5-a7aaad6d122e-tmp\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.076292 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076247    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ee7722b6-b03d-4160-89d5-a7aaad6d122e-snapshots\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.076292 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6cv7\" (UniqueName: \"kubernetes.io/projected/ee7722b6-b03d-4160-89d5-a7aaad6d122e-kube-api-access-l6cv7\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.076292 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076281    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7722b6-b03d-4160-89d5-a7aaad6d122e-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.076292 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076299    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7722b6-b03d-4160-89d5-a7aaad6d122e-serving-cert\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.076633 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076373    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:13.076633 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076454    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/87da33a9-c88f-4f09-ac99-9bc91c35d81f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:13.076633 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7722b6-b03d-4160-89d5-a7aaad6d122e-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.076633 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076554    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79ldn\" (UniqueName: \"kubernetes.io/projected/87da33a9-c88f-4f09-ac99-9bc91c35d81f-kube-api-access-79ldn\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:13.076633 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076575    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z58q\" (UniqueName: \"kubernetes.io/projected/1079be64-5c3f-4df5-9437-034a2eab9c8e-kube-api-access-9z58q\") pod \"network-check-source-869dcfcdcd-6c5jb\" (UID: \"1079be64-5c3f-4df5-9437-034a2eab9c8e\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb"
Jul 09 18:26:13.076898 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:13.076672    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:13.076898 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.076733    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7722b6-b03d-4160-89d5-a7aaad6d122e-tmp\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.076898 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:13.076777    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls podName:87da33a9-c88f-4f09-ac99-9bc91c35d81f nodeName:}" failed. No retries permitted until 2026-07-09 18:26:13.576755925 +0000 UTC m=+50.712573380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zkbpj" (UID: "87da33a9-c88f-4f09-ac99-9bc91c35d81f") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:13.077102 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.077065    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ee7722b6-b03d-4160-89d5-a7aaad6d122e-snapshots\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.077226 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.077205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7722b6-b03d-4160-89d5-a7aaad6d122e-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.077313 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.077291    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/87da33a9-c88f-4f09-ac99-9bc91c35d81f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:13.077397 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.077378    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7722b6-b03d-4160-89d5-a7aaad6d122e-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.078789 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.078772    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7722b6-b03d-4160-89d5-a7aaad6d122e-serving-cert\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.086004 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.085980    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79ldn\" (UniqueName: \"kubernetes.io/projected/87da33a9-c88f-4f09-ac99-9bc91c35d81f-kube-api-access-79ldn\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:13.086086 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.086047    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6cv7\" (UniqueName: \"kubernetes.io/projected/ee7722b6-b03d-4160-89d5-a7aaad6d122e-kube-api-access-l6cv7\") pod \"insights-operator-68d9dd94cf-ctcrt\" (UID: \"ee7722b6-b03d-4160-89d5-a7aaad6d122e\") " pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.086086 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.086052    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z58q\" (UniqueName: \"kubernetes.io/projected/1079be64-5c3f-4df5-9437-034a2eab9c8e-kube-api-access-9z58q\") pod \"network-check-source-869dcfcdcd-6c5jb\" (UID: \"1079be64-5c3f-4df5-9437-034a2eab9c8e\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb"
Jul 09 18:26:13.186323 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.186284    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt"
Jul 09 18:26:13.212146 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.212115    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb"
Jul 09 18:26:13.349041 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.349013    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-ctcrt"]
Jul 09 18:26:13.355913 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:13.355871    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7722b6_b03d_4160_89d5_a7aaad6d122e.slice/crio-5f07ba13b6710618d27c37336436732b0ea45d0d8a58f320fbf21ec08ed219a3 WatchSource:0}: Error finding container 5f07ba13b6710618d27c37336436732b0ea45d0d8a58f320fbf21ec08ed219a3: Status 404 returned error can't find the container with id 5f07ba13b6710618d27c37336436732b0ea45d0d8a58f320fbf21ec08ed219a3
Jul 09 18:26:13.394251 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.394219    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb"]
Jul 09 18:26:13.398896 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:13.398873    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1079be64_5c3f_4df5_9437_034a2eab9c8e.slice/crio-4458c5f774d8ef25e626f9bed3e6d0aa8c642a0327540126f9a16d86c42ac856 WatchSource:0}: Error finding container 4458c5f774d8ef25e626f9bed3e6d0aa8c642a0327540126f9a16d86c42ac856: Status 404 returned error can't find the container with id 4458c5f774d8ef25e626f9bed3e6d0aa8c642a0327540126f9a16d86c42ac856
Jul 09 18:26:13.479938 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.479902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:13.480123 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:13.480051    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:26:13.480123 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:13.480112    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls podName:bc82e223-fb55-4ae1-8b06-011f014427f1 nodeName:}" failed. No retries permitted until 2026-07-09 18:26:14.480098025 +0000 UTC m=+51.615915477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls") pod "cluster-samples-operator-65db858894-gfmvf" (UID: "bc82e223-fb55-4ae1-8b06-011f014427f1") : secret "samples-operator-tls" not found
Jul 09 18:26:13.548854 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.548816    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt" event={"ID":"ee7722b6-b03d-4160-89d5-a7aaad6d122e","Type":"ContainerStarted","Data":"5f07ba13b6710618d27c37336436732b0ea45d0d8a58f320fbf21ec08ed219a3"}
Jul 09 18:26:13.549843 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.549825    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb" event={"ID":"1079be64-5c3f-4df5-9437-034a2eab9c8e","Type":"ContainerStarted","Data":"4458c5f774d8ef25e626f9bed3e6d0aa8c642a0327540126f9a16d86c42ac856"}
Jul 09 18:26:13.580294 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:13.580268    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:13.580442 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:13.580422    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:13.580494 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:13.580486    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls podName:87da33a9-c88f-4f09-ac99-9bc91c35d81f nodeName:}" failed. No retries permitted until 2026-07-09 18:26:14.580472309 +0000 UTC m=+51.716289759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zkbpj" (UID: "87da33a9-c88f-4f09-ac99-9bc91c35d81f") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:14.488893 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:14.488852    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:14.489344 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:14.489030    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:26:14.489344 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:14.489112    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls podName:bc82e223-fb55-4ae1-8b06-011f014427f1 nodeName:}" failed. No retries permitted until 2026-07-09 18:26:16.489089829 +0000 UTC m=+53.624907279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls") pod "cluster-samples-operator-65db858894-gfmvf" (UID: "bc82e223-fb55-4ae1-8b06-011f014427f1") : secret "samples-operator-tls" not found
Jul 09 18:26:14.590404 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:14.590360    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:14.590574 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:14.590524    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:14.590673 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:14.590597    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls podName:87da33a9-c88f-4f09-ac99-9bc91c35d81f nodeName:}" failed. No retries permitted until 2026-07-09 18:26:16.590576965 +0000 UTC m=+53.726394427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zkbpj" (UID: "87da33a9-c88f-4f09-ac99-9bc91c35d81f") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:16.506440 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:16.506398    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:16.506908 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:16.506549    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:26:16.506908 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:16.506615    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls podName:bc82e223-fb55-4ae1-8b06-011f014427f1 nodeName:}" failed. No retries permitted until 2026-07-09 18:26:20.506598776 +0000 UTC m=+57.642416230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls") pod "cluster-samples-operator-65db858894-gfmvf" (UID: "bc82e223-fb55-4ae1-8b06-011f014427f1") : secret "samples-operator-tls" not found
Jul 09 18:26:16.607320 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:16.607285    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:16.607470 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:16.607441    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:16.607516 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:16.607506    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls podName:87da33a9-c88f-4f09-ac99-9bc91c35d81f nodeName:}" failed. No retries permitted until 2026-07-09 18:26:20.607489886 +0000 UTC m=+57.743307339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zkbpj" (UID: "87da33a9-c88f-4f09-ac99-9bc91c35d81f") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:18.562383 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:18.562293    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb" event={"ID":"1079be64-5c3f-4df5-9437-034a2eab9c8e","Type":"ContainerStarted","Data":"ef519ae8faa64540b70494dbcafd68e76ed8fc056f980fc0ec5d4137dc4b3da7"}
Jul 09 18:26:18.564235 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:18.564209    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt" event={"ID":"ee7722b6-b03d-4160-89d5-a7aaad6d122e","Type":"ContainerStarted","Data":"03da3a4f2d07902e7556288c19fff3ad7039dc31e5742174766016be62f33467"}
Jul 09 18:26:18.610091 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:18.610034    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6c5jb" podStartSLOduration=1.750624736 podStartE2EDuration="6.610016279s" totalImagesPullingTime="4.859391543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:18.587306279 +0000 UTC m=+55.723123751" watchObservedRunningTime="2026-07-09 18:26:18.610016279 +0000 UTC m=+55.745833750"
Jul 09 18:26:18.610613 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:18.610590    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt" podStartSLOduration=2.033965536 podStartE2EDuration="6.610582267s" totalImagesPullingTime="4.576616731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:18.609454241 +0000 UTC m=+55.745271724" watchObservedRunningTime="2026-07-09 18:26:18.610582267 +0000 UTC m=+55.746399738"
Jul 09 18:26:19.078865 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.078838    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-c5x6g"]
Jul 09 18:26:19.081038 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.081019    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.083831 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.083550    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 18:26:19.083831 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.083673    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 18:26:19.083831 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.083785    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tvx4s\""
Jul 09 18:26:19.093843 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.093822    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c5x6g"]
Jul 09 18:26:19.128032 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.128005    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.128153 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.128094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krvcp\" (UniqueName: \"kubernetes.io/projected/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-api-access-krvcp\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.128153 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.128120    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a04edb26-2f53-47f1-b1cd-b94142d242dd-data-volume\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.128262 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.128145    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.128317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.128290    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a04edb26-2f53-47f1-b1cd-b94142d242dd-crio-socket\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.228853 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.228820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a04edb26-2f53-47f1-b1cd-b94142d242dd-crio-socket\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.228980 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.228878    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.228980 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.228944    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krvcp\" (UniqueName: \"kubernetes.io/projected/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-api-access-krvcp\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.228980 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.228966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a04edb26-2f53-47f1-b1cd-b94142d242dd-data-volume\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.229335 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.228991    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.229335 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.229034    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 18:26:19.229335 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.229051    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a04edb26-2f53-47f1-b1cd-b94142d242dd-crio-socket\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.229335 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.229090    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 18:26:19.229335 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.229119    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm podName:a04edb26-2f53-47f1-b1cd-b94142d242dd nodeName:}" failed. No retries permitted until 2026-07-09 18:26:19.729098356 +0000 UTC m=+56.864915809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5x6g" (UID: "a04edb26-2f53-47f1-b1cd-b94142d242dd") : configmap "kube-rbac-proxy" not found
Jul 09 18:26:19.229335 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.229138    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls podName:a04edb26-2f53-47f1-b1cd-b94142d242dd nodeName:}" failed. No retries permitted until 2026-07-09 18:26:19.729128775 +0000 UTC m=+56.864946231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5x6g" (UID: "a04edb26-2f53-47f1-b1cd-b94142d242dd") : secret "insights-runtime-extractor-tls" not found
Jul 09 18:26:19.229335 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.229312    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a04edb26-2f53-47f1-b1cd-b94142d242dd-data-volume\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.239369 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.239347    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krvcp\" (UniqueName: \"kubernetes.io/projected/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-api-access-krvcp\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.531786 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.531730    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:19.532036 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.531932    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:26:19.532036 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.531955    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bdc4656cf-frhhm: secret "image-registry-tls" not found
Jul 09 18:26:19.532036 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.532031    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls podName:32c563e6-5352-453d-b256-e3ed5819a18c nodeName:}" failed. No retries permitted until 2026-07-09 18:26:35.532009183 +0000 UTC m=+72.667826649 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls") pod "image-registry-5bdc4656cf-frhhm" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c") : secret "image-registry-tls" not found
Jul 09 18:26:19.632632 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.632597    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:19.633023 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.632831    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:26:19.633023 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.632929    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls podName:6f37ae45-4062-40db-9fc7-6fcec23b742e nodeName:}" failed. No retries permitted until 2026-07-09 18:26:35.632889156 +0000 UTC m=+72.768706614 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls") pod "dns-default-xv799" (UID: "6f37ae45-4062-40db-9fc7-6fcec23b742e") : secret "dns-default-metrics-tls" not found
Jul 09 18:26:19.733175 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.733143    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.733319 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:19.733206    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:19.733319 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.733257    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 18:26:19.733319 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.733315    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm podName:a04edb26-2f53-47f1-b1cd-b94142d242dd nodeName:}" failed. No retries permitted until 2026-07-09 18:26:20.733301313 +0000 UTC m=+57.869118762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5x6g" (UID: "a04edb26-2f53-47f1-b1cd-b94142d242dd") : configmap "kube-rbac-proxy" not found
Jul 09 18:26:19.733424 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.733327    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 18:26:19.733424 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:19.733362    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls podName:a04edb26-2f53-47f1-b1cd-b94142d242dd nodeName:}" failed. No retries permitted until 2026-07-09 18:26:20.733351725 +0000 UTC m=+57.869169175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5x6g" (UID: "a04edb26-2f53-47f1-b1cd-b94142d242dd") : secret "insights-runtime-extractor-tls" not found
Jul 09 18:26:20.540371 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:20.540339    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:20.540533 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:20.540484    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:26:20.540578 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:20.540546    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls podName:bc82e223-fb55-4ae1-8b06-011f014427f1 nodeName:}" failed. No retries permitted until 2026-07-09 18:26:28.54053185 +0000 UTC m=+65.676349303 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls") pod "cluster-samples-operator-65db858894-gfmvf" (UID: "bc82e223-fb55-4ae1-8b06-011f014427f1") : secret "samples-operator-tls" not found
Jul 09 18:26:20.641007 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:20.640978    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:20.641348 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:20.641117    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:20.641348 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:20.641174    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls podName:87da33a9-c88f-4f09-ac99-9bc91c35d81f nodeName:}" failed. No retries permitted until 2026-07-09 18:26:28.641160721 +0000 UTC m=+65.776978174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zkbpj" (UID: "87da33a9-c88f-4f09-ac99-9bc91c35d81f") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:20.741390 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:20.741344    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:20.741558 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:20.741435    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:20.741558 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:20.741468    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 18:26:20.741558 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:20.741528    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm podName:a04edb26-2f53-47f1-b1cd-b94142d242dd nodeName:}" failed. No retries permitted until 2026-07-09 18:26:22.741513284 +0000 UTC m=+59.877330737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5x6g" (UID: "a04edb26-2f53-47f1-b1cd-b94142d242dd") : configmap "kube-rbac-proxy" not found
Jul 09 18:26:20.741686 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:20.741578    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 18:26:20.741686 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:20.741627    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls podName:a04edb26-2f53-47f1-b1cd-b94142d242dd nodeName:}" failed. No retries permitted until 2026-07-09 18:26:22.741615264 +0000 UTC m=+59.877432714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5x6g" (UID: "a04edb26-2f53-47f1-b1cd-b94142d242dd") : secret "insights-runtime-extractor-tls" not found
Jul 09 18:26:21.679851 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:21.679825    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2mb6q_59b49275-d607-420f-acaf-10d3b9843b44/dns-node-resolver/0.log"
Jul 09 18:26:22.759243 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:22.759201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:22.759627 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:22.759280    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:22.759627 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:22.759338    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 18:26:22.759627 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:22.759384    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 18:26:22.759627 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:22.759402    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm podName:a04edb26-2f53-47f1-b1cd-b94142d242dd nodeName:}" failed. No retries permitted until 2026-07-09 18:26:26.759387562 +0000 UTC m=+63.895205016 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5x6g" (UID: "a04edb26-2f53-47f1-b1cd-b94142d242dd") : configmap "kube-rbac-proxy" not found
Jul 09 18:26:22.759627 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:22.759423    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls podName:a04edb26-2f53-47f1-b1cd-b94142d242dd nodeName:}" failed. No retries permitted until 2026-07-09 18:26:26.759412167 +0000 UTC m=+63.895229616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5x6g" (UID: "a04edb26-2f53-47f1-b1cd-b94142d242dd") : secret "insights-runtime-extractor-tls" not found
Jul 09 18:26:22.888403 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:22.888373    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-slptj_07a65c98-48c0-4cd8-a737-5908696e29b6/node-ca/0.log"
Jul 09 18:26:23.876589 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:23.876561    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xdlwn_274a4b22-0f83-4378-a747-f6a741e5abf3/serve-healthcheck-canary/0.log"
Jul 09 18:26:26.790380 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:26.790346    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:26.790861 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:26.790411    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:26.790861 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:26.790481    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 18:26:26.790861 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:26.790508    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 18:26:26.790861 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:26.790559    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm podName:a04edb26-2f53-47f1-b1cd-b94142d242dd nodeName:}" failed. No retries permitted until 2026-07-09 18:26:34.790539899 +0000 UTC m=+71.926357351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5x6g" (UID: "a04edb26-2f53-47f1-b1cd-b94142d242dd") : configmap "kube-rbac-proxy" not found
Jul 09 18:26:26.790861 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:26.790594    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls podName:a04edb26-2f53-47f1-b1cd-b94142d242dd nodeName:}" failed. No retries permitted until 2026-07-09 18:26:34.790583069 +0000 UTC m=+71.926400528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5x6g" (UID: "a04edb26-2f53-47f1-b1cd-b94142d242dd") : secret "insights-runtime-extractor-tls" not found
Jul 09 18:26:28.605175 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:28.605146    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:28.607582 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:28.607558    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc82e223-fb55-4ae1-8b06-011f014427f1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gfmvf\" (UID: \"bc82e223-fb55-4ae1-8b06-011f014427f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:28.705398 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:28.705371    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:28.705518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:28.705502    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4c5k9\""
Jul 09 18:26:28.705557 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:28.705505    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:28.705589 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:28.705573    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls podName:87da33a9-c88f-4f09-ac99-9bc91c35d81f nodeName:}" failed. No retries permitted until 2026-07-09 18:26:44.705558169 +0000 UTC m=+81.841375622 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zkbpj" (UID: "87da33a9-c88f-4f09-ac99-9bc91c35d81f") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:26:28.712799 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:28.712778    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"
Jul 09 18:26:28.831192 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:28.831166    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf"]
Jul 09 18:26:29.007959 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.007932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:26:29.008055 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.008000    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:26:29.013145 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.013130    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 18:26:29.013699 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.013677    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 18:26:29.018869 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:29.018851    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 18:26:29.018952 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:29.018901    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs podName:735bceb1-4f3b-4163-a164-a70f462b5aac nodeName:}" failed. No retries permitted until 2026-07-09 18:27:33.018886108 +0000 UTC m=+130.154703561 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs") pod "network-metrics-daemon-llr8j" (UID: "735bceb1-4f3b-4163-a164-a70f462b5aac") : secret "metrics-daemon-secret" not found
Jul 09 18:26:29.020783 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.020761    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9876487f-ee1d-4875-940e-d0032bc17427-original-pull-secret\") pod \"global-pull-secret-syncer-7m46n\" (UID: \"9876487f-ee1d-4875-940e-d0032bc17427\") " pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:26:29.062527 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.062505    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7m46n"
Jul 09 18:26:29.178510 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.178482    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7m46n"]
Jul 09 18:26:29.181557 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:29.181533    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9876487f_ee1d_4875_940e_d0032bc17427.slice/crio-21e951ab219fde778aa7644ef3b856d7e0f6dc4cb942f1457f34bca7f46598fa WatchSource:0}: Error finding container 21e951ab219fde778aa7644ef3b856d7e0f6dc4cb942f1457f34bca7f46598fa: Status 404 returned error can't find the container with id 21e951ab219fde778aa7644ef3b856d7e0f6dc4cb942f1457f34bca7f46598fa
Jul 09 18:26:29.210045 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.210025    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qkml\" (UniqueName: \"kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml\") pod \"network-check-target-xx64v\" (UID: \"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0\") " pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:26:29.212238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.212221    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qkml\" (UniqueName: \"kubernetes.io/projected/962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0-kube-api-access-2qkml\") pod \"network-check-target-xx64v\" (UID: \"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0\") " pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:26:29.268211 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.268189    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qx56j\""
Jul 09 18:26:29.276219 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.276201    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:26:29.396697 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.396620    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xx64v"]
Jul 09 18:26:29.399214 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:29.399189    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962e4fae_b0a1_4ca4_9a3e_99aaf06dc7c0.slice/crio-965b64fd399da924e9c2b8f0a70a6a2412b01c7736635b971cab8fb028a9123c WatchSource:0}: Error finding container 965b64fd399da924e9c2b8f0a70a6a2412b01c7736635b971cab8fb028a9123c: Status 404 returned error can't find the container with id 965b64fd399da924e9c2b8f0a70a6a2412b01c7736635b971cab8fb028a9123c
Jul 09 18:26:29.514570 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.514551    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chbbq"
Jul 09 18:26:29.592271 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.592247    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7m46n" event={"ID":"9876487f-ee1d-4875-940e-d0032bc17427","Type":"ContainerStarted","Data":"21e951ab219fde778aa7644ef3b856d7e0f6dc4cb942f1457f34bca7f46598fa"}
Jul 09 18:26:29.593422 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.593399    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf" event={"ID":"bc82e223-fb55-4ae1-8b06-011f014427f1","Type":"ContainerStarted","Data":"05ebc6dcf9bfefee48ff8b3fa5816861fb4cbe04d00760a895bb5d59edf76b3f"}
Jul 09 18:26:29.594632 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.594614    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xx64v" event={"ID":"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0","Type":"ContainerStarted","Data":"28407eefaa12ca95feda125fda6c89adc54525a4f4b5c6a3f577468a7fcd1310"}
Jul 09 18:26:29.594744 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.594661    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xx64v" event={"ID":"962e4fae-b0a1-4ca4-9a3e-99aaf06dc7c0","Type":"ContainerStarted","Data":"965b64fd399da924e9c2b8f0a70a6a2412b01c7736635b971cab8fb028a9123c"}
Jul 09 18:26:29.594744 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:29.594702    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:26:31.603395 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:31.603317    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf" event={"ID":"bc82e223-fb55-4ae1-8b06-011f014427f1","Type":"ContainerStarted","Data":"7549f90ac543ed8a9a6fce42635d3767f996a989fa12ee120355aaa954c0eee9"}
Jul 09 18:26:31.603395 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:31.603369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf" event={"ID":"bc82e223-fb55-4ae1-8b06-011f014427f1","Type":"ContainerStarted","Data":"8384b7ab592f1c9b5864a994b71bf59054a90a2e335e171ba269ab5c8d708ca3"}
Jul 09 18:26:31.626513 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:31.626471    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xx64v" podStartSLOduration=68.62636716 podStartE2EDuration="1m8.62636716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:29.613192911 +0000 UTC m=+66.749010383" watchObservedRunningTime="2026-07-09 18:26:31.62636716 +0000 UTC m=+68.762184662"
Jul 09 18:26:31.626845 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:31.626819    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gfmvf" podStartSLOduration=17.255622513 podStartE2EDuration="19.626810421s" totalImagesPullingTime="2.371187908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:31.624488192 +0000 UTC m=+68.760305665" watchObservedRunningTime="2026-07-09 18:26:31.626810421 +0000 UTC m=+68.762627891"
Jul 09 18:26:34.853450 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:34.853411    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:34.853894 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:34.853491    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:34.854033 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:34.854015    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a04edb26-2f53-47f1-b1cd-b94142d242dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:34.856035 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:34.856008    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a04edb26-2f53-47f1-b1cd-b94142d242dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5x6g\" (UID: \"a04edb26-2f53-47f1-b1cd-b94142d242dd\") " pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:34.995320 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:34.995294    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tvx4s\""
Jul 09 18:26:35.003550 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.003518    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c5x6g"
Jul 09 18:26:35.559495 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.559465    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:35.562315 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.562282    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") pod \"image-registry-5bdc4656cf-frhhm\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") " pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:35.568707 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.568680    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lxd8z\""
Jul 09 18:26:35.576970 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.576949    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:35.660371 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.660346    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:35.662715 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.662694    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f37ae45-4062-40db-9fc7-6fcec23b742e-metrics-tls\") pod \"dns-default-xv799\" (UID: \"6f37ae45-4062-40db-9fc7-6fcec23b742e\") " pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:35.948729 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.948708    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jnv68\""
Jul 09 18:26:35.955734 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.955708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:35.967166 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.967093    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c5x6g"]
Jul 09 18:26:35.968981 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:35.968911    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bdc4656cf-frhhm"]
Jul 09 18:26:35.971719 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:35.971561    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04edb26_2f53_47f1_b1cd_b94142d242dd.slice/crio-7b44be071b1c21af796007dd39b45ba648d69c86fd1ee2973ecc5b15dad98559 WatchSource:0}: Error finding container 7b44be071b1c21af796007dd39b45ba648d69c86fd1ee2973ecc5b15dad98559: Status 404 returned error can't find the container with id 7b44be071b1c21af796007dd39b45ba648d69c86fd1ee2973ecc5b15dad98559
Jul 09 18:26:35.975316 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:35.973910    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c563e6_5352_453d_b256_e3ed5819a18c.slice/crio-cd7b4526bf64e116026902a0e282cc4ba1adfa2e3da790377a57e7df1b310d65 WatchSource:0}: Error finding container cd7b4526bf64e116026902a0e282cc4ba1adfa2e3da790377a57e7df1b310d65: Status 404 returned error can't find the container with id cd7b4526bf64e116026902a0e282cc4ba1adfa2e3da790377a57e7df1b310d65
Jul 09 18:26:36.108162 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:36.108079    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xv799"]
Jul 09 18:26:36.111688 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:36.111662    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f37ae45_4062_40db_9fc7_6fcec23b742e.slice/crio-12692439395acb6846be0980ebf4831090bd619c2292c375a1c9c5f91c62becf WatchSource:0}: Error finding container 12692439395acb6846be0980ebf4831090bd619c2292c375a1c9c5f91c62becf: Status 404 returned error can't find the container with id 12692439395acb6846be0980ebf4831090bd619c2292c375a1c9c5f91c62becf
Jul 09 18:26:36.624316 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:36.624278    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm" event={"ID":"32c563e6-5352-453d-b256-e3ed5819a18c","Type":"ContainerStarted","Data":"dacd8572bf3eadbba85a3d103badf71c7c8e19a0b294ca62c4e675857d962914"}
Jul 09 18:26:36.624493 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:36.624341    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm" event={"ID":"32c563e6-5352-453d-b256-e3ed5819a18c","Type":"ContainerStarted","Data":"cd7b4526bf64e116026902a0e282cc4ba1adfa2e3da790377a57e7df1b310d65"}
Jul 09 18:26:36.625091 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:36.625052    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:36.628521 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:36.628207    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7m46n" event={"ID":"9876487f-ee1d-4875-940e-d0032bc17427","Type":"ContainerStarted","Data":"3361f143bc833fb199f53ab78ac9f505dd78077ca839a065fe508321b795a624"}
Jul 09 18:26:36.630788 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:36.630723    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5x6g" event={"ID":"a04edb26-2f53-47f1-b1cd-b94142d242dd","Type":"ContainerStarted","Data":"7b44be071b1c21af796007dd39b45ba648d69c86fd1ee2973ecc5b15dad98559"}
Jul 09 18:26:36.634201 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:36.634132    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xv799" event={"ID":"6f37ae45-4062-40db-9fc7-6fcec23b742e","Type":"ContainerStarted","Data":"12692439395acb6846be0980ebf4831090bd619c2292c375a1c9c5f91c62becf"}
Jul 09 18:26:36.648025 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:36.647827    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm" podStartSLOduration=73.647810261 podStartE2EDuration="1m13.647810261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:36.645756859 +0000 UTC m=+73.781574345" watchObservedRunningTime="2026-07-09 18:26:36.647810261 +0000 UTC m=+73.783627735"
Jul 09 18:26:36.664475 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:36.663822    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7m46n" podStartSLOduration=67.01787162 podStartE2EDuration="1m13.663805299s" totalImagesPullingTime="6.645933679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:36.663535537 +0000 UTC m=+73.799353010" watchObservedRunningTime="2026-07-09 18:26:36.663805299 +0000 UTC m=+73.799622763"
Jul 09 18:26:39.931403 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:39.931364    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"]
Jul 09 18:26:39.940797 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:39.940770    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:39.943245 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:39.943221    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 09 18:26:39.943905 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:39.943885    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 09 18:26:39.943998 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:39.943912    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 09 18:26:39.943998 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:39.943912    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 09 18:26:39.947560 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:39.947539    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"]
Jul 09 18:26:40.102006 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.101978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a4694357-56ca-4274-a0e7-97ab6d3f9327-klusterlet-config\") pod \"klusterlet-addon-workmgr-74bc6b6f8d-9ndxl\" (UID: \"a4694357-56ca-4274-a0e7-97ab6d3f9327\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:40.102162 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.102018    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw4v4\" (UniqueName: \"kubernetes.io/projected/a4694357-56ca-4274-a0e7-97ab6d3f9327-kube-api-access-vw4v4\") pod \"klusterlet-addon-workmgr-74bc6b6f8d-9ndxl\" (UID: \"a4694357-56ca-4274-a0e7-97ab6d3f9327\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:40.102162 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.102065    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4694357-56ca-4274-a0e7-97ab6d3f9327-tmp\") pod \"klusterlet-addon-workmgr-74bc6b6f8d-9ndxl\" (UID: \"a4694357-56ca-4274-a0e7-97ab6d3f9327\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:40.202906 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.202829    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a4694357-56ca-4274-a0e7-97ab6d3f9327-klusterlet-config\") pod \"klusterlet-addon-workmgr-74bc6b6f8d-9ndxl\" (UID: \"a4694357-56ca-4274-a0e7-97ab6d3f9327\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:40.202906 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.202873    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw4v4\" (UniqueName: \"kubernetes.io/projected/a4694357-56ca-4274-a0e7-97ab6d3f9327-kube-api-access-vw4v4\") pod \"klusterlet-addon-workmgr-74bc6b6f8d-9ndxl\" (UID: \"a4694357-56ca-4274-a0e7-97ab6d3f9327\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:40.203113 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.202921    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4694357-56ca-4274-a0e7-97ab6d3f9327-tmp\") pod \"klusterlet-addon-workmgr-74bc6b6f8d-9ndxl\" (UID: \"a4694357-56ca-4274-a0e7-97ab6d3f9327\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:40.203398 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.203364    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4694357-56ca-4274-a0e7-97ab6d3f9327-tmp\") pod \"klusterlet-addon-workmgr-74bc6b6f8d-9ndxl\" (UID: \"a4694357-56ca-4274-a0e7-97ab6d3f9327\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:40.205657 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.205621    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a4694357-56ca-4274-a0e7-97ab6d3f9327-klusterlet-config\") pod \"klusterlet-addon-workmgr-74bc6b6f8d-9ndxl\" (UID: \"a4694357-56ca-4274-a0e7-97ab6d3f9327\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:40.221146 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.221120    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw4v4\" (UniqueName: \"kubernetes.io/projected/a4694357-56ca-4274-a0e7-97ab6d3f9327-kube-api-access-vw4v4\") pod \"klusterlet-addon-workmgr-74bc6b6f8d-9ndxl\" (UID: \"a4694357-56ca-4274-a0e7-97ab6d3f9327\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:40.244497 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.244471    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-mw59t"]
Jul 09 18:26:40.247345 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.247327    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-mw59t"
Jul 09 18:26:40.249491 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.249471    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-sc7jv\""
Jul 09 18:26:40.249491 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.249485    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 18:26:40.249634 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.249471    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 18:26:40.252039 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.252021    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:40.257832 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.257812    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-mw59t"]
Jul 09 18:26:40.404747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.404569    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zgrg\" (UniqueName: \"kubernetes.io/projected/54a5b984-6393-45da-8d25-0d86810f0c18-kube-api-access-6zgrg\") pod \"downloads-68c9cbb497-mw59t\" (UID: \"54a5b984-6393-45da-8d25-0d86810f0c18\") " pod="openshift-console/downloads-68c9cbb497-mw59t"
Jul 09 18:26:40.487023 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.486993    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"]
Jul 09 18:26:40.505683 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.505640    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zgrg\" (UniqueName: \"kubernetes.io/projected/54a5b984-6393-45da-8d25-0d86810f0c18-kube-api-access-6zgrg\") pod \"downloads-68c9cbb497-mw59t\" (UID: \"54a5b984-6393-45da-8d25-0d86810f0c18\") " pod="openshift-console/downloads-68c9cbb497-mw59t"
Jul 09 18:26:40.517176 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.517151    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zgrg\" (UniqueName: \"kubernetes.io/projected/54a5b984-6393-45da-8d25-0d86810f0c18-kube-api-access-6zgrg\") pod \"downloads-68c9cbb497-mw59t\" (UID: \"54a5b984-6393-45da-8d25-0d86810f0c18\") " pod="openshift-console/downloads-68c9cbb497-mw59t"
Jul 09 18:26:40.559879 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.559856    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-mw59t"
Jul 09 18:26:40.650933 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.650895    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5x6g" event={"ID":"a04edb26-2f53-47f1-b1cd-b94142d242dd","Type":"ContainerStarted","Data":"44bfee508e84cb2027f357480fec275436dd5529eef26b3607e7b2a2b73abf9c"}
Jul 09 18:26:40.652131 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.652108    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl" event={"ID":"a4694357-56ca-4274-a0e7-97ab6d3f9327","Type":"ContainerStarted","Data":"65d572fbc336df8b18c72374bf01550ad460751afd1df25497ce4678aba93601"}
Jul 09 18:26:40.653946 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.653927    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xv799" event={"ID":"6f37ae45-4062-40db-9fc7-6fcec23b742e","Type":"ContainerStarted","Data":"37fbb93138e74c66f38dec2353ed164a870f458bbd6b8d80d1fa70128554a9e3"}
Jul 09 18:26:40.654026 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.653968    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xv799" event={"ID":"6f37ae45-4062-40db-9fc7-6fcec23b742e","Type":"ContainerStarted","Data":"f514d94bff7b08a64c38fcc430a9d318e8b2df5934deb9500996c5b02ef5b1f4"}
Jul 09 18:26:40.654194 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.654173    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:40.674040 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.674000    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xv799" podStartSLOduration=33.437102612 podStartE2EDuration="37.673986032s" totalImagesPullingTime="4.23688342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:40.672766044 +0000 UTC m=+77.808583527" watchObservedRunningTime="2026-07-09 18:26:40.673986032 +0000 UTC m=+77.809803503"
Jul 09 18:26:40.679668 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:40.679625    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-mw59t"]
Jul 09 18:26:40.683592 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:40.683560    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a5b984_6393_45da_8d25_0d86810f0c18.slice/crio-fa49bf8b498754d2ec928c78bc62f03e4667a31af6ea4ca76d9d9e47cfd6664a WatchSource:0}: Error finding container fa49bf8b498754d2ec928c78bc62f03e4667a31af6ea4ca76d9d9e47cfd6664a: Status 404 returned error can't find the container with id fa49bf8b498754d2ec928c78bc62f03e4667a31af6ea4ca76d9d9e47cfd6664a
Jul 09 18:26:41.660636 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:41.660591    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-mw59t" event={"ID":"54a5b984-6393-45da-8d25-0d86810f0c18","Type":"ContainerStarted","Data":"fa49bf8b498754d2ec928c78bc62f03e4667a31af6ea4ca76d9d9e47cfd6664a"}
Jul 09 18:26:41.664388 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:41.664364    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5x6g" event={"ID":"a04edb26-2f53-47f1-b1cd-b94142d242dd","Type":"ContainerStarted","Data":"cb5459e24b9f19af441bafbf587ea253314ee8f471533cb0d7e468f029884ca7"}
Jul 09 18:26:44.679121 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:44.679019    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5x6g" event={"ID":"a04edb26-2f53-47f1-b1cd-b94142d242dd","Type":"ContainerStarted","Data":"057ef10e0585628427d87bf37d5701587cd5a588bcaeee78208401eff10f1217"}
Jul 09 18:26:44.702250 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:44.702195    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-c5x6g" podStartSLOduration=17.474144697 podStartE2EDuration="25.70217443s" totalImagesPullingTime="8.228029733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:26:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:44.701692719 +0000 UTC m=+81.837510191" watchObservedRunningTime="2026-07-09 18:26:44.70217443 +0000 UTC m=+81.837991906"
Jul 09 18:26:44.742003 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:44.741968    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:44.744580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:44.744558    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/87da33a9-c88f-4f09-ac99-9bc91c35d81f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zkbpj\" (UID: \"87da33a9-c88f-4f09-ac99-9bc91c35d81f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:44.999999 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:44.999920    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nb5gr\""
Jul 09 18:26:45.007356 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:45.007332    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"
Jul 09 18:26:45.684546 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:45.684379    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl" event={"ID":"a4694357-56ca-4274-a0e7-97ab6d3f9327","Type":"ContainerStarted","Data":"feeb44258b8a8fb5ddb81f1c3a2efa28216f248e0e92a251a88ca00885ef7561"}
Jul 09 18:26:45.684926 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:45.684629    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:45.685706 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:45.685676    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl" podUID="a4694357-56ca-4274-a0e7-97ab6d3f9327" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.14:8000/readyz\": dial tcp 10.132.0.14:8000: connect: connection refused"
Jul 09 18:26:45.701468 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:45.701447    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj"]
Jul 09 18:26:45.704381 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:45.704344    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl" podStartSLOduration=1.580099696 podStartE2EDuration="6.704330837s" totalImagesPullingTime="5.124231141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:45.703036597 +0000 UTC m=+82.838854069" watchObservedRunningTime="2026-07-09 18:26:45.704330837 +0000 UTC m=+82.840148308"
Jul 09 18:26:45.708956 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:45.708931    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87da33a9_c88f_4f09_ac99_9bc91c35d81f.slice/crio-ce715bf83f54851eed1951805d73723573ee3199ff1d88d142e8a7b9a0152b86 WatchSource:0}: Error finding container ce715bf83f54851eed1951805d73723573ee3199ff1d88d142e8a7b9a0152b86: Status 404 returned error can't find the container with id ce715bf83f54851eed1951805d73723573ee3199ff1d88d142e8a7b9a0152b86
Jul 09 18:26:46.688776 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:46.688739    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj" event={"ID":"87da33a9-c88f-4f09-ac99-9bc91c35d81f","Type":"ContainerStarted","Data":"ce715bf83f54851eed1951805d73723573ee3199ff1d88d142e8a7b9a0152b86"}
Jul 09 18:26:46.690059 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:46.690034    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74bc6b6f8d-9ndxl"
Jul 09 18:26:48.294449 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:48.294416    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"]
Jul 09 18:26:48.296369 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:48.296348    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"
Jul 09 18:26:48.298889 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:48.298861    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"prometheus-operator-admission-webhook-77d98bc786-v9wdw\" is forbidden: User \"system:node:ip-10-0-135-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-28.ec2.internal' and this object" podUID="a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"
Jul 09 18:26:48.298889 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:48.298870    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-admission-webhook-dockercfg-gbl5f\" is forbidden: User \"system:node:ip-10-0-135-28.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-28.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gbl5f\"" type="*v1.Secret"
Jul 09 18:26:48.299623 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:48.299598    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-admission-webhook-tls\" is forbidden: User \"system:node:ip-10-0-135-28.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-28.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" type="*v1.Secret"
Jul 09 18:26:48.325069 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:48.325051    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"]
Jul 09 18:26:48.468455 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:48.468421    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-v9wdw\" (UID: \"a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"
Jul 09 18:26:48.569555 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:48.569473    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-v9wdw\" (UID: \"a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"
Jul 09 18:26:48.698732 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:48.698684    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj" event={"ID":"87da33a9-c88f-4f09-ac99-9bc91c35d81f","Type":"ContainerStarted","Data":"fc4203982818929ea9f587670c81266920a91c1eeab80b7b8972ff39af129ab7"}
Jul 09 18:26:48.731018 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:48.730972    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zkbpj" podStartSLOduration=34.735856951 podStartE2EDuration="36.730956748s" totalImagesPullingTime="1.995099797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:48.715282293 +0000 UTC m=+85.851099767" watchObservedRunningTime="2026-07-09 18:26:48.730956748 +0000 UTC m=+85.866774220"
Jul 09 18:26:49.223953 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:49.223924    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gbl5f\""
Jul 09 18:26:49.570020 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:49.569938    2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: failed to sync secret cache: timed out waiting for the condition
Jul 09 18:26:49.570414 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:49.570023    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63-tls-certificates podName:a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63 nodeName:}" failed. No retries permitted until 2026-07-09 18:26:50.070005467 +0000 UTC m=+87.205822920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-v9wdw" (UID: "a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63") : failed to sync secret cache: timed out waiting for the condition
Jul 09 18:26:49.679540 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:49.679517    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 09 18:26:50.080994 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.080955    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-v9wdw\" (UID: \"a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"
Jul 09 18:26:50.083700 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.083672    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-v9wdw\" (UID: \"a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"
Jul 09 18:26:50.105603 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.105577    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"
Jul 09 18:26:50.259983 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.259949    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"]
Jul 09 18:26:50.265913 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:50.265881    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0d2a938_0ca6_4a72_94f8_3a6b9b1f1f63.slice/crio-e719c25b17ae7d935cdbd218cb75854661970088ab592758e1d4b6645f945586 WatchSource:0}: Error finding container e719c25b17ae7d935cdbd218cb75854661970088ab592758e1d4b6645f945586: Status 404 returned error can't find the container with id e719c25b17ae7d935cdbd218cb75854661970088ab592758e1d4b6645f945586
Jul 09 18:26:50.356815 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.356751    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8dc75bf8-xhpvz"]
Jul 09 18:26:50.363768 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.363748    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.366435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.366406    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 18:26:50.366588 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.366554    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 18:26:50.366588 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.366554    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 18:26:50.367566 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.367548    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 18:26:50.367667 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.367574    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 18:26:50.367667 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.367658    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8bstw\""
Jul 09 18:26:50.372982 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.372956    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8dc75bf8-xhpvz"]
Jul 09 18:26:50.484661 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.484626    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qln6\" (UniqueName: \"kubernetes.io/projected/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-kube-api-access-8qln6\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.484825 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.484681    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-oauth-config\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.484825 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.484724    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-service-ca\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.484825 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.484775    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-oauth-serving-cert\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.484825 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.484803    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-serving-cert\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.484825 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.484825    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-config\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.585179 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.585146    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qln6\" (UniqueName: \"kubernetes.io/projected/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-kube-api-access-8qln6\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.585622 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.585186    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-oauth-config\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.585622 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.585221    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-service-ca\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.585622 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.585281    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-oauth-serving-cert\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.585622 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.585325    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-serving-cert\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.585622 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.585360    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-config\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.586197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.586172    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-config\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.586296 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.586195    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-oauth-serving-cert\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.586296 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.586255    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-service-ca\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.588088 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.588065    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-oauth-config\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.588245 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.588227    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-serving-cert\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.595165 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.595141    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qln6\" (UniqueName: \"kubernetes.io/projected/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-kube-api-access-8qln6\") pod \"console-8dc75bf8-xhpvz\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") " pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.666346 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.666327    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xv799"
Jul 09 18:26:50.674413 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.674398    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:26:50.706493 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.706470    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw" event={"ID":"a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63","Type":"ContainerStarted","Data":"e719c25b17ae7d935cdbd218cb75854661970088ab592758e1d4b6645f945586"}
Jul 09 18:26:50.833638 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:50.833611    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8dc75bf8-xhpvz"]
Jul 09 18:26:50.836839 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:50.836810    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd569a641_bc3e_4721_bc1f_e6b5bfd1f60c.slice/crio-be95305e4847a192ed9062e2154e630c13364f6877539391c770414c5fdd86c7 WatchSource:0}: Error finding container be95305e4847a192ed9062e2154e630c13364f6877539391c770414c5fdd86c7: Status 404 returned error can't find the container with id be95305e4847a192ed9062e2154e630c13364f6877539391c770414c5fdd86c7
Jul 09 18:26:51.710454 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:51.710423    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc75bf8-xhpvz" event={"ID":"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c","Type":"ContainerStarted","Data":"be95305e4847a192ed9062e2154e630c13364f6877539391c770414c5fdd86c7"}
Jul 09 18:26:52.716412 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:52.716376    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw" event={"ID":"a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63","Type":"ContainerStarted","Data":"90f6e55bbd72c8a99a0c2774926ba65f07d1e2ac02b820676f587c9fb5715752"}
Jul 09 18:26:52.716890 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:52.716775    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"
Jul 09 18:26:52.722967 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:52.722909    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw"
Jul 09 18:26:52.735287 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:52.735236    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-v9wdw" podStartSLOduration=3.131557265 podStartE2EDuration="4.735217308s" totalImagesPullingTime="1.603660043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:52.733105713 +0000 UTC m=+89.868923187" watchObservedRunningTime="2026-07-09 18:26:52.735217308 +0000 UTC m=+89.871034760"
Jul 09 18:26:54.952991 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:54.952957    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f9486c8d7-srgk2"]
Jul 09 18:26:54.958863 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:54.958837    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:54.967178 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:54.967143    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 18:26:54.969064 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:54.969038    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f9486c8d7-srgk2"]
Jul 09 18:26:55.021595 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.021574    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-oauth-config\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.021773 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.021607    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-console-config\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.021773 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.021623    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-trusted-ca-bundle\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.021773 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.021637    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-oauth-serving-cert\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.021773 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.021701    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt8ds\" (UniqueName: \"kubernetes.io/projected/59557c96-fd65-44e1-8d99-a42a2dac5632-kube-api-access-mt8ds\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.021773 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.021735    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-service-ca\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.021958 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.021777    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-serving-cert\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.122751 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.122718    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-service-ca\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.122916 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.122781    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-serving-cert\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.122916 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.122841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-oauth-config\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.122916 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.122880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-console-config\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.122916 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.122904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-trusted-ca-bundle\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.123140 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.122929    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-oauth-serving-cert\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.123140 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.122968    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt8ds\" (UniqueName: \"kubernetes.io/projected/59557c96-fd65-44e1-8d99-a42a2dac5632-kube-api-access-mt8ds\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.123484 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.123459    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-service-ca\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.123743 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.123701    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-console-config\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.123997 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.123976    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-oauth-serving-cert\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.124140 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.124118    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-trusted-ca-bundle\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.126503 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.126482    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-oauth-config\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.126641 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.126622    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-serving-cert\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.132371 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.132352    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt8ds\" (UniqueName: \"kubernetes.io/projected/59557c96-fd65-44e1-8d99-a42a2dac5632-kube-api-access-mt8ds\") pod \"console-6f9486c8d7-srgk2\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") " pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.270225 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.270146    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:26:55.398999 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.398962    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f9486c8d7-srgk2"]
Jul 09 18:26:55.402128 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:26:55.402096    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59557c96_fd65_44e1_8d99_a42a2dac5632.slice/crio-0f0323621d6f069ffef3029f01bf93b10e7a058625685b130a53bcc2020830bb WatchSource:0}: Error finding container 0f0323621d6f069ffef3029f01bf93b10e7a058625685b130a53bcc2020830bb: Status 404 returned error can't find the container with id 0f0323621d6f069ffef3029f01bf93b10e7a058625685b130a53bcc2020830bb
Jul 09 18:26:55.582115 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.582033    2566 patch_prober.go:28] interesting pod/image-registry-5bdc4656cf-frhhm container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 18:26:55.582115 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.582095    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm" podUID="32c563e6-5352-453d-b256-e3ed5819a18c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 18:26:55.731046 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.730794    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc75bf8-xhpvz" event={"ID":"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c","Type":"ContainerStarted","Data":"e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc"}
Jul 09 18:26:55.733225 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.733188    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9486c8d7-srgk2" event={"ID":"59557c96-fd65-44e1-8d99-a42a2dac5632","Type":"ContainerStarted","Data":"702309a3b789206922743a423d202e4dcd1acdb01653dec40e06d35170cbb340"}
Jul 09 18:26:55.733354 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.733232    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9486c8d7-srgk2" event={"ID":"59557c96-fd65-44e1-8d99-a42a2dac5632","Type":"ContainerStarted","Data":"0f0323621d6f069ffef3029f01bf93b10e7a058625685b130a53bcc2020830bb"}
Jul 09 18:26:55.751528 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.751479    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8dc75bf8-xhpvz" podStartSLOduration=1.5955751089999999 podStartE2EDuration="5.75146373s" totalImagesPullingTime="4.155888621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:55.750201627 +0000 UTC m=+92.886019099" watchObservedRunningTime="2026-07-09 18:26:55.75146373 +0000 UTC m=+92.887281201"
Jul 09 18:26:55.774823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:55.774778    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f9486c8d7-srgk2" podStartSLOduration=1.774757506 podStartE2EDuration="1.774757506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:55.773952128 +0000 UTC m=+92.909769603" watchObservedRunningTime="2026-07-09 18:26:55.774757506 +0000 UTC m=+92.910574979"
Jul 09 18:26:58.647825 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.647787    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:26:58.778353 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.778323    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"]
Jul 09 18:26:58.781253 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.781228    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.783766 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.783740    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7qw5j\""
Jul 09 18:26:58.783933 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.783873    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 18:26:58.784131 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.784112    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 18:26:58.786920 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.786895    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 18:26:58.801485 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.801460    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"]
Jul 09 18:26:58.836848 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.836824    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-76jqh"]
Jul 09 18:26:58.839661 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.839461    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.842941 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.842915    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 18:26:58.843085 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.843017    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 18:26:58.843224 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.843207    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xcq5v\""
Jul 09 18:26:58.843302 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.843283    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 18:26:58.848837 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.848812    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.848946 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.848846    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-root\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.848946 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.848877    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-accelerators-collector-config\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.849067 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.848963    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-metrics-client-ca\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.849067 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.849002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-wtmp\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.849067 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.849017    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gls8s\" (UniqueName: \"kubernetes.io/projected/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-kube-api-access-gls8s\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.849067 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.849039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68k9m\" (UniqueName: \"kubernetes.io/projected/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-kube-api-access-68k9m\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.849067 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.849066    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-textfile\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.849308 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.849092    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.849308 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.849151    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-sys\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.849308 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.849191    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-tls\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.849308 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.849255    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.849308 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.849285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.949866 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.949836    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-wtmp\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950021 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.949871    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gls8s\" (UniqueName: \"kubernetes.io/projected/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-kube-api-access-gls8s\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950021 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.949896    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68k9m\" (UniqueName: \"kubernetes.io/projected/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-kube-api-access-68k9m\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.950133 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-textfile\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950133 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950103    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-wtmp\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950133 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950119    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.950277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950151    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-sys\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-tls\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950212    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-sys\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.950456 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950299    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950456 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:58.950303    2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 09 18:26:58.950456 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:26:58.950360    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-tls podName:f10348ed-d7cb-4ea4-a599-ea5613e9abb7 nodeName:}" failed. No retries permitted until 2026-07-09 18:26:59.450341573 +0000 UTC m=+96.586159023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-tls") pod "node-exporter-76jqh" (UID: "f10348ed-d7cb-4ea4-a599-ea5613e9abb7") : secret "node-exporter-tls" not found
Jul 09 18:26:58.950456 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950374    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.950456 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950392    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-root\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950456 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950426    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-accelerators-collector-config\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950788 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950465    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-metrics-client-ca\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950788 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950495    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-textfile\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950788 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950513    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-root\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.950986 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950826    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.951040 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.950995    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-metrics-client-ca\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.951094 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.951044    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-accelerators-collector-config\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.953132 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.953107    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:58.953359 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.953337    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.953436 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.953360    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.975155 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.975131    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68k9m\" (UniqueName: \"kubernetes.io/projected/f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa-kube-api-access-68k9m\") pod \"openshift-state-metrics-84bd68b896-ctdf7\" (UID: \"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:58.975853 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:58.975835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gls8s\" (UniqueName: \"kubernetes.io/projected/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-kube-api-access-gls8s\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:59.092663 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.092586    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"
Jul 09 18:26:59.454764 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.454729    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-tls\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:59.457277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.457249    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f10348ed-d7cb-4ea4-a599-ea5613e9abb7-node-exporter-tls\") pod \"node-exporter-76jqh\" (UID: \"f10348ed-d7cb-4ea4-a599-ea5613e9abb7\") " pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:59.753184 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.753111    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-76jqh"
Jul 09 18:26:59.816096 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.816065    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:26:59.827129 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.827103    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.829737 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.829704    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 18:26:59.829737 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.829705    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 18:26:59.829916 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.829858    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 18:26:59.829916 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.829880    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 18:26:59.830402 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.830382    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sk6tg\""
Jul 09 18:26:59.830402 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.830401    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 18:26:59.830569 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.830461    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 18:26:59.830569 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.830490    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 18:26:59.830688 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.830631    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 18:26:59.830891 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.830864    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 18:26:59.840044 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.840025    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:26:59.858281 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858257    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.858427 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858296    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.858427 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.858427 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858412    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5w4m\" (UniqueName: \"kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-kube-api-access-l5w4m\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.858614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858448    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-volume\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.858614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858476    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-out\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.858614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858501    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.858614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858524    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.858614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858563    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.858614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858590    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.858614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858614    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-web-config\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.859083 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858664    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.859083 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.858730    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.959779 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.959742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5w4m\" (UniqueName: \"kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-kube-api-access-l5w4m\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.959938 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.959788    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-volume\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.959938 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.959810    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-out\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.959938 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.959832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.959938 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.959855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.959938 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.959880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.959938 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.959900    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.959938 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.959925    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-web-config\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.961392 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.960614    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.961392 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.960718    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.961392 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.960731    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.961392 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.960832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.961392 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.960870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.961392 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.960902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.961392 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.961066    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.961844 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.961718    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.962942 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.962916    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-out\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.963506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.963460    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-volume\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.964166 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.964138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.965404 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.964542    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.965404 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.965008    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.965404 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.965024    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-web-config\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.965404 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.965054    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.965404 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.965358    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.966078 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.966059    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:26:59.976370 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:26:59.976331    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5w4m\" (UniqueName: \"kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-kube-api-access-l5w4m\") pod \"alertmanager-main-0\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:27:00.140466 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:00.140382    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:27:00.601220 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:00.601100    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xx64v"
Jul 09 18:27:00.675070 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:00.675031    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:27:00.675236 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:00.675094    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:27:00.681364 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:00.681340    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:27:00.758411 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:00.758381    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:27:02.684846 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:02.684814    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bdc4656cf-frhhm"]
Jul 09 18:27:03.332747 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:27:03.332713    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf10348ed_d7cb_4ea4_a599_ea5613e9abb7.slice/crio-2a5ec1e45747ac050dc6585a98d4e60c17fa37e3f712209124405f5c7822f60d WatchSource:0}: Error finding container 2a5ec1e45747ac050dc6585a98d4e60c17fa37e3f712209124405f5c7822f60d: Status 404 returned error can't find the container with id 2a5ec1e45747ac050dc6585a98d4e60c17fa37e3f712209124405f5c7822f60d
Jul 09 18:27:03.467006 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.466976    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7"]
Jul 09 18:27:03.497370 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.497352    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:27:03.618423 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:27:03.618347    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a4b2f7_81a2_4d82_9944_f0c0e01fbffa.slice/crio-1095e9d0a3b8e4f7c43b7cc5acc591605aeb987d457b644926f255043d6dbb1d WatchSource:0}: Error finding container 1095e9d0a3b8e4f7c43b7cc5acc591605aeb987d457b644926f255043d6dbb1d: Status 404 returned error can't find the container with id 1095e9d0a3b8e4f7c43b7cc5acc591605aeb987d457b644926f255043d6dbb1d
Jul 09 18:27:03.619124 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:27:03.619094    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda92cf30_e6db_419b_9b62_ccdadf71f2ae.slice/crio-8e83f002c5cdfbea80fb5a9e4e3bd3cd432763546f087e6c1c86d83d7a50f986 WatchSource:0}: Error finding container 8e83f002c5cdfbea80fb5a9e4e3bd3cd432763546f087e6c1c86d83d7a50f986: Status 404 returned error can't find the container with id 8e83f002c5cdfbea80fb5a9e4e3bd3cd432763546f087e6c1c86d83d7a50f986
Jul 09 18:27:03.768190 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.768164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76jqh" event={"ID":"f10348ed-d7cb-4ea4-a599-ea5613e9abb7","Type":"ContainerStarted","Data":"2a5ec1e45747ac050dc6585a98d4e60c17fa37e3f712209124405f5c7822f60d"}
Jul 09 18:27:03.770201 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.770173    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerStarted","Data":"8e83f002c5cdfbea80fb5a9e4e3bd3cd432763546f087e6c1c86d83d7a50f986"}
Jul 09 18:27:03.772790 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.772709    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-mw59t" event={"ID":"54a5b984-6393-45da-8d25-0d86810f0c18","Type":"ContainerStarted","Data":"113e5f499696ac09d8cfac4266887248431b4ffec335c570df78a513c7960305"}
Jul 09 18:27:03.773657 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.773623    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-mw59t"
Jul 09 18:27:03.775817 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.775722    2566 patch_prober.go:28] interesting pod/downloads-68c9cbb497-mw59t container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused" start-of-body=
Jul 09 18:27:03.775817 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.775765    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-mw59t" podUID="54a5b984-6393-45da-8d25-0d86810f0c18" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused"
Jul 09 18:27:03.777242 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.777216    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7" event={"ID":"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa","Type":"ContainerStarted","Data":"db0df6409fea0464b28e0c743c11dd2013046285b2426bf04bb0821bf54d19cd"}
Jul 09 18:27:03.777355 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.777273    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7" event={"ID":"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa","Type":"ContainerStarted","Data":"1095e9d0a3b8e4f7c43b7cc5acc591605aeb987d457b644926f255043d6dbb1d"}
Jul 09 18:27:03.815028 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:03.814106    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-mw59t" podStartSLOduration=0.804177899 podStartE2EDuration="23.814085914s" totalImagesPullingTime="23.009908015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:27:03.811806035 +0000 UTC m=+100.947623557" watchObservedRunningTime="2026-07-09 18:27:03.814085914 +0000 UTC m=+100.949903387"
Jul 09 18:27:04.788435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:04.788396    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7" event={"ID":"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa","Type":"ContainerStarted","Data":"ef2f3be4d7c525ad2f23421b0d6446232f33a6ae012c0599cc8b57021094d19a"}
Jul 09 18:27:04.797279 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:04.797196    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-mw59t"
Jul 09 18:27:05.256055 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.255746    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:27:05.288676 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.286639    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:27:05.288676 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.286708    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:27:05.288676 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.286781    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:27:05.288676 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.286795    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.288676 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.286806    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:27:05.292269 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.291843    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 18:27:05.292269 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.292076    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1n95bka1a46mu\""
Jul 09 18:27:05.292734 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.292546    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 18:27:05.297136 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.297117    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 18:27:05.297523 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.297468    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.297694    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.297859    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.297873    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.298424    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.298565    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6pqfg\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.298869    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.299006    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.299142    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.299235    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.299430    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.299508    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 18:27:05.299682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.299568    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 18:27:05.305369 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.300958    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 18:27:05.315541 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315514    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.315664 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315601    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config-out\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.315664 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315657    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.315776 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315692    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.315776 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315721    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.315776 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315747    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-web-config\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.315776 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315773    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgvv7\" (UniqueName: \"kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-kube-api-access-cgvv7\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315830    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315871    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315901    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.315978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.316001    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.316024    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.316046    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.316106    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.316132    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.316155    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.316178    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.316277 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.316201    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.410662 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.410610    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8dc75bf8-xhpvz"]
Jul 09 18:27:05.417699 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.417664    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.417862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.417712    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.417862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.417739    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.417862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.417764    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.417862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.417806    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.417862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.417832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.417862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.417854    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418174 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.417877    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418174 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.417912    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418174 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.417966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418174 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.418026    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config-out\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418174 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.418057    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418174 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.418092    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418174 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.418118    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418174 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.418148    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-web-config\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418174 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.418174    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvv7\" (UniqueName: \"kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-kube-api-access-cgvv7\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418609 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.418202    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418609 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.418244    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418609 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.418274    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.418609 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.418303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.423599 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.423569    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.423724 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.423577    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.424226 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.424201    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.424902 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.424875    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.425752 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.425722    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.426132 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.426041    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.429145 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.426534    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.429145 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.427060    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.429145 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.427841    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.434534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.434366    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.434534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.434374    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.434534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.434527    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.435232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.434909    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.435232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.435091    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.435232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.435113    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config-out\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.435232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.435185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.436005 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.435940    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-web-config\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.436005 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.435946    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.436534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.436516    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.438976 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.438945    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgvv7\" (UniqueName: \"kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-kube-api-access-cgvv7\") pod \"prometheus-k8s-0\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.609156 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.609009    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:05.800166 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.800132    2566 generic.go:356] "Generic (PLEG): container finished" podID="f10348ed-d7cb-4ea4-a599-ea5613e9abb7" containerID="5dff4b4dc245dd1dfb647246af3f27dc19416119019b93d75a483c4d7d5e56f9" exitCode=0
Jul 09 18:27:05.800620 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:05.800236    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76jqh" event={"ID":"f10348ed-d7cb-4ea4-a599-ea5613e9abb7","Type":"ContainerDied","Data":"5dff4b4dc245dd1dfb647246af3f27dc19416119019b93d75a483c4d7d5e56f9"}
Jul 09 18:27:06.503461 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.503216    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:27:06.513103 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:27:06.509851    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8171757_5f32_4b76_8e74_8dbdc43dcfcb.slice/crio-3e589e04c8e4f70ac0c005a756b8f467972c27dd128b5ba9399efbd659785a77 WatchSource:0}: Error finding container 3e589e04c8e4f70ac0c005a756b8f467972c27dd128b5ba9399efbd659785a77: Status 404 returned error can't find the container with id 3e589e04c8e4f70ac0c005a756b8f467972c27dd128b5ba9399efbd659785a77
Jul 09 18:27:06.806297 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.806210    2566 generic.go:356] "Generic (PLEG): container finished" podID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerID="797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563" exitCode=0
Jul 09 18:27:06.806747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.806302    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerDied","Data":"797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563"}
Jul 09 18:27:06.812290 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.812158    2566 generic.go:356] "Generic (PLEG): container finished" podID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerID="62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332" exitCode=0
Jul 09 18:27:06.812384 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.812283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerDied","Data":"62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332"}
Jul 09 18:27:06.812384 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.812357    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerStarted","Data":"3e589e04c8e4f70ac0c005a756b8f467972c27dd128b5ba9399efbd659785a77"}
Jul 09 18:27:06.815389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.815280    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7" event={"ID":"f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa","Type":"ContainerStarted","Data":"0146bae974ae0fc4727516f6275705b144810acf93bd2e5810c65d2c38772ae2"}
Jul 09 18:27:06.817953 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.817875    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76jqh" event={"ID":"f10348ed-d7cb-4ea4-a599-ea5613e9abb7","Type":"ContainerStarted","Data":"88857454b2cabd608406d24a8ff8f333af7486b2e772f3bcf741da733598a7e4"}
Jul 09 18:27:06.817953 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.817922    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76jqh" event={"ID":"f10348ed-d7cb-4ea4-a599-ea5613e9abb7","Type":"ContainerStarted","Data":"953d16158d9ff8837fb634a97e98162dfe1bb02c74f42300373c3524fe026fb2"}
Jul 09 18:27:06.851409 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.851362    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-76jqh" podStartSLOduration=7.354437557 podStartE2EDuration="8.851338212s" totalImagesPullingTime="1.385470469s" totalInitContainerRuntime="111.430186ms" isStatefulPod=true podCreationTimestamp="2026-07-09 18:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:27:06.850265049 +0000 UTC m=+103.986082521" watchObservedRunningTime="2026-07-09 18:27:06.851338212 +0000 UTC m=+103.987155685"
Jul 09 18:27:06.868754 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:06.868715    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ctdf7" podStartSLOduration=6.319436884 podStartE2EDuration="8.86870226s" totalImagesPullingTime="2.549265376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:27:06.867403624 +0000 UTC m=+104.003221097" watchObservedRunningTime="2026-07-09 18:27:06.86870226 +0000 UTC m=+104.004519746"
Jul 09 18:27:09.835501 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:09.835471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerStarted","Data":"754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14"}
Jul 09 18:27:09.835940 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:09.835548    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerStarted","Data":"ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628"}
Jul 09 18:27:10.845536 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:10.845499    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerStarted","Data":"46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587"}
Jul 09 18:27:10.846013 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:10.845582    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerStarted","Data":"fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433"}
Jul 09 18:27:10.846013 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:10.845670    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerStarted","Data":"aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca"}
Jul 09 18:27:12.865225 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:12.865189    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerStarted","Data":"299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62"}
Jul 09 18:27:12.865601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:12.865283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerStarted","Data":"e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94"}
Jul 09 18:27:13.871710 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:13.871675    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerStarted","Data":"a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c"}
Jul 09 18:27:13.898617 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:13.898558    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.879762941 podStartE2EDuration="14.898539086s" totalImagesPullingTime="7.996193983s" totalInitContainerRuntime="22.582162ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:27:13.896500734 +0000 UTC m=+111.032318242" watchObservedRunningTime="2026-07-09 18:27:13.898539086 +0000 UTC m=+111.034356559"
Jul 09 18:27:15.884148 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:15.884121    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerStarted","Data":"c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce"}
Jul 09 18:27:15.884550 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:15.884213    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerStarted","Data":"ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad"}
Jul 09 18:27:15.884550 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:15.884240    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerStarted","Data":"d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586"}
Jul 09 18:27:16.891328 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:16.891294    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerStarted","Data":"2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea"}
Jul 09 18:27:16.925508 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:16.925463    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.780331871 podStartE2EDuration="11.925448205s" totalImagesPullingTime="8.121775561s" totalInitContainerRuntime="23.340773ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:27:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:27:16.921593784 +0000 UTC m=+114.057411256" watchObservedRunningTime="2026-07-09 18:27:16.925448205 +0000 UTC m=+114.061265698"
Jul 09 18:27:20.609890 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:20.609858    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:27:22.845391 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:22.845357    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f9486c8d7-srgk2"]
Jul 09 18:27:27.709542 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:27.709492    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm" podUID="32c563e6-5352-453d-b256-e3ed5819a18c" containerName="registry" containerID="cri-o://dacd8572bf3eadbba85a3d103badf71c7c8e19a0b294ca62c4e675857d962914" gracePeriod=30
Jul 09 18:27:27.940140 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:27.940109    2566 generic.go:356] "Generic (PLEG): container finished" podID="32c563e6-5352-453d-b256-e3ed5819a18c" containerID="dacd8572bf3eadbba85a3d103badf71c7c8e19a0b294ca62c4e675857d962914" exitCode=0
Jul 09 18:27:27.940264 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:27.940202    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm" event={"ID":"32c563e6-5352-453d-b256-e3ed5819a18c","Type":"ContainerDied","Data":"dacd8572bf3eadbba85a3d103badf71c7c8e19a0b294ca62c4e675857d962914"}
Jul 09 18:27:27.989936 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:27.989910    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:27:28.121008 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.120984    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") pod \"32c563e6-5352-453d-b256-e3ed5819a18c\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") "
Jul 09 18:27:28.121131 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.121017    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-trusted-ca\") pod \"32c563e6-5352-453d-b256-e3ed5819a18c\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") "
Jul 09 18:27:28.121131 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.121036    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-bound-sa-token\") pod \"32c563e6-5352-453d-b256-e3ed5819a18c\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") "
Jul 09 18:27:28.121131 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.121068    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32c563e6-5352-453d-b256-e3ed5819a18c-ca-trust-extracted\") pod \"32c563e6-5352-453d-b256-e3ed5819a18c\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") "
Jul 09 18:27:28.121131 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.121095    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-image-registry-private-configuration\") pod \"32c563e6-5352-453d-b256-e3ed5819a18c\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") "
Jul 09 18:27:28.121131 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.121121    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltsdz\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-kube-api-access-ltsdz\") pod \"32c563e6-5352-453d-b256-e3ed5819a18c\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") "
Jul 09 18:27:28.121387 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.121146    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-registry-certificates\") pod \"32c563e6-5352-453d-b256-e3ed5819a18c\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") "
Jul 09 18:27:28.121387 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.121165    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-installation-pull-secrets\") pod \"32c563e6-5352-453d-b256-e3ed5819a18c\" (UID: \"32c563e6-5352-453d-b256-e3ed5819a18c\") "
Jul 09 18:27:28.121512 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.121477    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "32c563e6-5352-453d-b256-e3ed5819a18c" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:27:28.121774 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.121746    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "32c563e6-5352-453d-b256-e3ed5819a18c" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:27:28.123441 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.123409    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "32c563e6-5352-453d-b256-e3ed5819a18c" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:27:28.123546 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.123528    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "32c563e6-5352-453d-b256-e3ed5819a18c" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:27:28.123709 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.123660    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "32c563e6-5352-453d-b256-e3ed5819a18c" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:27:28.123788 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.123713    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "32c563e6-5352-453d-b256-e3ed5819a18c" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:27:28.123850 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.123800    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-kube-api-access-ltsdz" (OuterVolumeSpecName: "kube-api-access-ltsdz") pod "32c563e6-5352-453d-b256-e3ed5819a18c" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c"). InnerVolumeSpecName "kube-api-access-ltsdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:27:28.130155 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.130130    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c563e6-5352-453d-b256-e3ed5819a18c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "32c563e6-5352-453d-b256-e3ed5819a18c" (UID: "32c563e6-5352-453d-b256-e3ed5819a18c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:27:28.222224 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.222175    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-registry-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:28.222224 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.222194    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-trusted-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:28.222224 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.222206    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-bound-sa-token\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:28.222224 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.222220    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32c563e6-5352-453d-b256-e3ed5819a18c-ca-trust-extracted\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:28.222395 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.222233    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-image-registry-private-configuration\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:28.222395 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.222248    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltsdz\" (UniqueName: \"kubernetes.io/projected/32c563e6-5352-453d-b256-e3ed5819a18c-kube-api-access-ltsdz\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:28.222395 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.222261    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32c563e6-5352-453d-b256-e3ed5819a18c-registry-certificates\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:28.222395 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.222276    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32c563e6-5352-453d-b256-e3ed5819a18c-installation-pull-secrets\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:28.949595 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.949563    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm" event={"ID":"32c563e6-5352-453d-b256-e3ed5819a18c","Type":"ContainerDied","Data":"cd7b4526bf64e116026902a0e282cc4ba1adfa2e3da790377a57e7df1b310d65"}
Jul 09 18:27:28.949595 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.949580    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bdc4656cf-frhhm"
Jul 09 18:27:28.950151 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.949669    2566 scope.go:117] "RemoveContainer" containerID="dacd8572bf3eadbba85a3d103badf71c7c8e19a0b294ca62c4e675857d962914"
Jul 09 18:27:28.971514 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.971493    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bdc4656cf-frhhm"]
Jul 09 18:27:28.977329 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:28.977308    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bdc4656cf-frhhm"]
Jul 09 18:27:29.358603 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:29.358548    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32c563e6-5352-453d-b256-e3ed5819a18c" path="/var/lib/kubelet/pods/32c563e6-5352-453d-b256-e3ed5819a18c/volumes"
Jul 09 18:27:30.443635 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.443578    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8dc75bf8-xhpvz" podUID="d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" containerName="console" containerID="cri-o://e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc" gracePeriod=15
Jul 09 18:27:30.730311 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.730288    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:27:30.842316 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842291    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-oauth-serving-cert\") pod \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") "
Jul 09 18:27:30.842455 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842347    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-config\") pod \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") "
Jul 09 18:27:30.842455 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842367    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-oauth-config\") pod \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") "
Jul 09 18:27:30.842568 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842473    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-serving-cert\") pod \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") "
Jul 09 18:27:30.842568 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842504    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qln6\" (UniqueName: \"kubernetes.io/projected/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-kube-api-access-8qln6\") pod \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") "
Jul 09 18:27:30.842568 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842538    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-service-ca\") pod \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\" (UID: \"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c\") "
Jul 09 18:27:30.842752 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842720    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" (UID: "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:27:30.842752 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842735    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-config" (OuterVolumeSpecName: "console-config") pod "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" (UID: "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:27:30.842942 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842918    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-oauth-serving-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:30.842942 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842942    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-config\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:30.843103 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.842965    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-service-ca" (OuterVolumeSpecName: "service-ca") pod "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" (UID: "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:27:30.844578 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.844556    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" (UID: "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:27:30.844882 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.844863    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" (UID: "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:27:30.844882 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.844867    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-kube-api-access-8qln6" (OuterVolumeSpecName: "kube-api-access-8qln6") pod "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" (UID: "d569a641-bc3e-4721-bc1f-e6b5bfd1f60c"). InnerVolumeSpecName "kube-api-access-8qln6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:27:30.943366 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.943345    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-oauth-config\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:30.943366 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.943363    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-console-serving-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:30.943484 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.943373    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qln6\" (UniqueName: \"kubernetes.io/projected/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-kube-api-access-8qln6\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:30.943484 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.943383    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:30.958906 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.958858    2566 generic.go:356] "Generic (PLEG): container finished" podID="d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" containerID="e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc" exitCode=0
Jul 09 18:27:30.958906 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.958902    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc75bf8-xhpvz"
Jul 09 18:27:30.959012 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.958939    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc75bf8-xhpvz" event={"ID":"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c","Type":"ContainerDied","Data":"e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc"}
Jul 09 18:27:30.959012 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.958993    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc75bf8-xhpvz" event={"ID":"d569a641-bc3e-4721-bc1f-e6b5bfd1f60c","Type":"ContainerDied","Data":"be95305e4847a192ed9062e2154e630c13364f6877539391c770414c5fdd86c7"}
Jul 09 18:27:30.959077 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.959018    2566 scope.go:117] "RemoveContainer" containerID="e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc"
Jul 09 18:27:30.966705 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.966690    2566 scope.go:117] "RemoveContainer" containerID="e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc"
Jul 09 18:27:30.966949 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:27:30.966931    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc\": container with ID starting with e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc not found: ID does not exist" containerID="e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc"
Jul 09 18:27:30.966997 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.966957    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc"} err="failed to get container status \"e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc\": rpc error: code = NotFound desc = could not find container \"e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc\": container with ID starting with e2f4b5fc5bba5b08181cfe5982f36f9788800b623133fc5bcc19e24bb33946fc not found: ID does not exist"
Jul 09 18:27:30.979096 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.979075    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8dc75bf8-xhpvz"]
Jul 09 18:27:30.983188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:30.983168    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8dc75bf8-xhpvz"]
Jul 09 18:27:31.358142 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:31.358075    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" path="/var/lib/kubelet/pods/d569a641-bc3e-4721-bc1f-e6b5bfd1f60c/volumes"
Jul 09 18:27:33.058852 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:33.058820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:27:33.061162 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:33.061136    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735bceb1-4f3b-4163-a164-a70f462b5aac-metrics-certs\") pod \"network-metrics-daemon-llr8j\" (UID: \"735bceb1-4f3b-4163-a164-a70f462b5aac\") " pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:27:33.172539 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:33.172517    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7pc9s\""
Jul 09 18:27:33.180681 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:33.180663    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-llr8j"
Jul 09 18:27:33.338883 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:33.338802    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-llr8j"]
Jul 09 18:27:33.341641 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:27:33.341603    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735bceb1_4f3b_4163_a164_a70f462b5aac.slice/crio-c55a7aec60bc409d16f47abe14653fcec6b9d4713a6d5d16579c79a10be35762 WatchSource:0}: Error finding container c55a7aec60bc409d16f47abe14653fcec6b9d4713a6d5d16579c79a10be35762: Status 404 returned error can't find the container with id c55a7aec60bc409d16f47abe14653fcec6b9d4713a6d5d16579c79a10be35762
Jul 09 18:27:33.973305 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:33.973269    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-llr8j" event={"ID":"735bceb1-4f3b-4163-a164-a70f462b5aac","Type":"ContainerStarted","Data":"c55a7aec60bc409d16f47abe14653fcec6b9d4713a6d5d16579c79a10be35762"}
Jul 09 18:27:34.978929 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:34.978900    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-llr8j" event={"ID":"735bceb1-4f3b-4163-a164-a70f462b5aac","Type":"ContainerStarted","Data":"f823913de1155cdd56e14b84cf9d2fb655fcd30906d2046597a4064a71f66769"}
Jul 09 18:27:35.984852 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:35.984814    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-llr8j" event={"ID":"735bceb1-4f3b-4163-a164-a70f462b5aac","Type":"ContainerStarted","Data":"5a1e56db89dccd8360641f2589734272b455c3c30e9abaea4a758461329afa1a"}
Jul 09 18:27:36.004522 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:36.004473    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-llr8j" podStartSLOduration=131.583348575 podStartE2EDuration="2m13.00445529s" totalImagesPullingTime="1.421106715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:27:36.003597841 +0000 UTC m=+133.139415313" watchObservedRunningTime="2026-07-09 18:27:36.00445529 +0000 UTC m=+133.140272763"
Jul 09 18:27:47.873183 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:47.873104    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f9486c8d7-srgk2" podUID="59557c96-fd65-44e1-8d99-a42a2dac5632" containerName="console" containerID="cri-o://702309a3b789206922743a423d202e4dcd1acdb01653dec40e06d35170cbb340" gracePeriod=15
Jul 09 18:27:48.032773 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.032743    2566 generic.go:356] "Generic (PLEG): container finished" podID="59557c96-fd65-44e1-8d99-a42a2dac5632" containerID="702309a3b789206922743a423d202e4dcd1acdb01653dec40e06d35170cbb340" exitCode=0
Jul 09 18:27:48.032884 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.032782    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9486c8d7-srgk2" event={"ID":"59557c96-fd65-44e1-8d99-a42a2dac5632","Type":"ContainerDied","Data":"702309a3b789206922743a423d202e4dcd1acdb01653dec40e06d35170cbb340"}
Jul 09 18:27:48.154880 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.154818    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:27:48.267914 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.267889    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-serving-cert\") pod \"59557c96-fd65-44e1-8d99-a42a2dac5632\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") "
Jul 09 18:27:48.268028 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.267927    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-oauth-config\") pod \"59557c96-fd65-44e1-8d99-a42a2dac5632\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") "
Jul 09 18:27:48.268028 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.267950    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-oauth-serving-cert\") pod \"59557c96-fd65-44e1-8d99-a42a2dac5632\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") "
Jul 09 18:27:48.268138 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.268076    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt8ds\" (UniqueName: \"kubernetes.io/projected/59557c96-fd65-44e1-8d99-a42a2dac5632-kube-api-access-mt8ds\") pod \"59557c96-fd65-44e1-8d99-a42a2dac5632\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") "
Jul 09 18:27:48.268196 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.268139    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-service-ca\") pod \"59557c96-fd65-44e1-8d99-a42a2dac5632\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") "
Jul 09 18:27:48.268196 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.268170    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-trusted-ca-bundle\") pod \"59557c96-fd65-44e1-8d99-a42a2dac5632\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") "
Jul 09 18:27:48.268299 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.268214    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-console-config\") pod \"59557c96-fd65-44e1-8d99-a42a2dac5632\" (UID: \"59557c96-fd65-44e1-8d99-a42a2dac5632\") "
Jul 09 18:27:48.268382 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.268358    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "59557c96-fd65-44e1-8d99-a42a2dac5632" (UID: "59557c96-fd65-44e1-8d99-a42a2dac5632"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:27:48.268534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.268489    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-oauth-serving-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:48.268534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.268489    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-service-ca" (OuterVolumeSpecName: "service-ca") pod "59557c96-fd65-44e1-8d99-a42a2dac5632" (UID: "59557c96-fd65-44e1-8d99-a42a2dac5632"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:27:48.268678 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.268596    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "59557c96-fd65-44e1-8d99-a42a2dac5632" (UID: "59557c96-fd65-44e1-8d99-a42a2dac5632"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:27:48.268740 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.268721    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-console-config" (OuterVolumeSpecName: "console-config") pod "59557c96-fd65-44e1-8d99-a42a2dac5632" (UID: "59557c96-fd65-44e1-8d99-a42a2dac5632"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:27:48.270054 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.270029    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "59557c96-fd65-44e1-8d99-a42a2dac5632" (UID: "59557c96-fd65-44e1-8d99-a42a2dac5632"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:27:48.270147 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.270098    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "59557c96-fd65-44e1-8d99-a42a2dac5632" (UID: "59557c96-fd65-44e1-8d99-a42a2dac5632"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:27:48.270219 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.270194    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59557c96-fd65-44e1-8d99-a42a2dac5632-kube-api-access-mt8ds" (OuterVolumeSpecName: "kube-api-access-mt8ds") pod "59557c96-fd65-44e1-8d99-a42a2dac5632" (UID: "59557c96-fd65-44e1-8d99-a42a2dac5632"). InnerVolumeSpecName "kube-api-access-mt8ds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:27:48.369786 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.369762    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:48.369786 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.369781    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-trusted-ca-bundle\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:48.369911 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.369790    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59557c96-fd65-44e1-8d99-a42a2dac5632-console-config\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:48.369911 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.369800    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-serving-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:48.369911 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.369807    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59557c96-fd65-44e1-8d99-a42a2dac5632-console-oauth-config\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:48.369911 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:48.369818    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt8ds\" (UniqueName: \"kubernetes.io/projected/59557c96-fd65-44e1-8d99-a42a2dac5632-kube-api-access-mt8ds\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:27:49.037858 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:49.037821    2566 generic.go:356] "Generic (PLEG): container finished" podID="ee7722b6-b03d-4160-89d5-a7aaad6d122e" containerID="03da3a4f2d07902e7556288c19fff3ad7039dc31e5742174766016be62f33467" exitCode=0
Jul 09 18:27:49.038265 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:49.037895    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt" event={"ID":"ee7722b6-b03d-4160-89d5-a7aaad6d122e","Type":"ContainerDied","Data":"03da3a4f2d07902e7556288c19fff3ad7039dc31e5742174766016be62f33467"}
Jul 09 18:27:49.038265 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:49.038192    2566 scope.go:117] "RemoveContainer" containerID="03da3a4f2d07902e7556288c19fff3ad7039dc31e5742174766016be62f33467"
Jul 09 18:27:49.039221 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:49.039203    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9486c8d7-srgk2"
Jul 09 18:27:49.039318 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:49.039242    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9486c8d7-srgk2" event={"ID":"59557c96-fd65-44e1-8d99-a42a2dac5632","Type":"ContainerDied","Data":"0f0323621d6f069ffef3029f01bf93b10e7a058625685b130a53bcc2020830bb"}
Jul 09 18:27:49.039318 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:49.039286    2566 scope.go:117] "RemoveContainer" containerID="702309a3b789206922743a423d202e4dcd1acdb01653dec40e06d35170cbb340"
Jul 09 18:27:49.069714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:49.069691    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f9486c8d7-srgk2"]
Jul 09 18:27:49.077714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:49.077693    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f9486c8d7-srgk2"]
Jul 09 18:27:49.362174 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:49.362096    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59557c96-fd65-44e1-8d99-a42a2dac5632" path="/var/lib/kubelet/pods/59557c96-fd65-44e1-8d99-a42a2dac5632/volumes"
Jul 09 18:27:50.044544 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:27:50.044512    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-ctcrt" event={"ID":"ee7722b6-b03d-4160-89d5-a7aaad6d122e","Type":"ContainerStarted","Data":"1b2c3d9454c3ac98a8f881e850c69911fd9dca24878023246e502d77cbc0f03c"}
Jul 09 18:28:05.609800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:05.609767    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:05.628496 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:05.628475    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:06.121622 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:06.121598    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:19.529601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:19.529569    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:28:19.529990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:19.529841    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="alertmanager" containerID="cri-o://ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628" gracePeriod=120
Jul 09 18:28:19.529990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:19.529912    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy-metric" containerID="cri-o://46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587" gracePeriod=120
Jul 09 18:28:19.529990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:19.529941    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy-web" containerID="cri-o://aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca" gracePeriod=120
Jul 09 18:28:19.530161 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:19.529980    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="config-reloader" containerID="cri-o://754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14" gracePeriod=120
Jul 09 18:28:19.530161 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:19.529993    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="prom-label-proxy" containerID="cri-o://a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c" gracePeriod=120
Jul 09 18:28:19.530161 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:19.529968    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy" containerID="cri-o://fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433" gracePeriod=120
Jul 09 18:28:20.162781 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.162747    2566 generic.go:356] "Generic (PLEG): container finished" podID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerID="a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c" exitCode=0
Jul 09 18:28:20.162781 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.162771    2566 generic.go:356] "Generic (PLEG): container finished" podID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerID="fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433" exitCode=0
Jul 09 18:28:20.162781 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.162780    2566 generic.go:356] "Generic (PLEG): container finished" podID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerID="754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14" exitCode=0
Jul 09 18:28:20.162781 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.162788    2566 generic.go:356] "Generic (PLEG): container finished" podID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerID="ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628" exitCode=0
Jul 09 18:28:20.163026 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.162821    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerDied","Data":"a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c"}
Jul 09 18:28:20.163026 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.162884    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerDied","Data":"fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433"}
Jul 09 18:28:20.163026 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.162915    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerDied","Data":"754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14"}
Jul 09 18:28:20.163026 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.162940    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerDied","Data":"ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628"}
Jul 09 18:28:20.774133 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.774110    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:20.800338 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800314    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800417 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800364    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-main-db\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800417 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800399    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-tls-assets\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800427    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-web-config\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800452    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-out\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800493    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-cluster-tls-config\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800676 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800530    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-volume\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800676 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800566    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-metric\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800676 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800601    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-trusted-ca-bundle\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800676 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800629    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-web\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800880 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800681    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-main-tls\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800880 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800707    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-metrics-client-ca\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800880 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800739    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5w4m\" (UniqueName: \"kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-kube-api-access-l5w4m\") pod \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\" (UID: \"da92cf30-e6db-419b-9b62-ccdadf71f2ae\") "
Jul 09 18:28:20.800880 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800740    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:28:20.801078 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.800970    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-main-db\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.801138 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.801090    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:20.801984 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.801941    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:20.803867 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.803811    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:28:20.803980 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.803893    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-kube-api-access-l5w4m" (OuterVolumeSpecName: "kube-api-access-l5w4m") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "kube-api-access-l5w4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:28:20.804046 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.803969    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:20.804254 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.804232    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-out" (OuterVolumeSpecName: "config-out") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:28:20.804487 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.804468    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-volume" (OuterVolumeSpecName: "config-volume") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:20.804714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.804687    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:20.805466 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.805446    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:20.806332 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.806297    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:20.808461 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.808436    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:20.813578 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.813554    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-web-config" (OuterVolumeSpecName: "web-config") pod "da92cf30-e6db-419b-9b62-ccdadf71f2ae" (UID: "da92cf30-e6db-419b-9b62-ccdadf71f2ae"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:20.901975 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.901912    2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-tls-assets\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.901975 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.901937    2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-web-config\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.901975 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.901949    2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-out\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.901975 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.901960    2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-cluster-tls-config\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.901975 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.901972    2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-config-volume\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.902210 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.901985    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.902210 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.901999    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.902210 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.902011    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.902210 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.902024    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-main-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.902210 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.902036    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da92cf30-e6db-419b-9b62-ccdadf71f2ae-metrics-client-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.902210 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.902048    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5w4m\" (UniqueName: \"kubernetes.io/projected/da92cf30-e6db-419b-9b62-ccdadf71f2ae-kube-api-access-l5w4m\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:20.902210 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:20.902060    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da92cf30-e6db-419b-9b62-ccdadf71f2ae-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:21.168718 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.168688    2566 generic.go:356] "Generic (PLEG): container finished" podID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerID="46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587" exitCode=0
Jul 09 18:28:21.168718 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.168710    2566 generic.go:356] "Generic (PLEG): container finished" podID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerID="aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca" exitCode=0
Jul 09 18:28:21.168912 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.168769    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerDied","Data":"46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587"}
Jul 09 18:28:21.168912 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.168802    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.168912 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.168835    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerDied","Data":"aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca"}
Jul 09 18:28:21.168912 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.168863    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da92cf30-e6db-419b-9b62-ccdadf71f2ae","Type":"ContainerDied","Data":"8e83f002c5cdfbea80fb5a9e4e3bd3cd432763546f087e6c1c86d83d7a50f986"}
Jul 09 18:28:21.168912 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.168900    2566 scope.go:117] "RemoveContainer" containerID="a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c"
Jul 09 18:28:21.176157 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.176141    2566 scope.go:117] "RemoveContainer" containerID="46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587"
Jul 09 18:28:21.182848 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.182834    2566 scope.go:117] "RemoveContainer" containerID="fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433"
Jul 09 18:28:21.189173 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.189154    2566 scope.go:117] "RemoveContainer" containerID="aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca"
Jul 09 18:28:21.196007 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.195991    2566 scope.go:117] "RemoveContainer" containerID="754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14"
Jul 09 18:28:21.202588 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.202572    2566 scope.go:117] "RemoveContainer" containerID="ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628"
Jul 09 18:28:21.212215 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.212188    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:28:21.213779 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.213759    2566 scope.go:117] "RemoveContainer" containerID="797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563"
Jul 09 18:28:21.218490 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.218470    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:28:21.221973 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.221955    2566 scope.go:117] "RemoveContainer" containerID="a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c"
Jul 09 18:28:21.222225 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:21.222205    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c\": container with ID starting with a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c not found: ID does not exist" containerID="a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c"
Jul 09 18:28:21.222271 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.222237    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c"} err="failed to get container status \"a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c\": rpc error: code = NotFound desc = could not find container \"a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c\": container with ID starting with a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c not found: ID does not exist"
Jul 09 18:28:21.222271 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.222255    2566 scope.go:117] "RemoveContainer" containerID="46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587"
Jul 09 18:28:21.222501 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:21.222480    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587\": container with ID starting with 46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587 not found: ID does not exist" containerID="46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587"
Jul 09 18:28:21.222575 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.222510    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587"} err="failed to get container status \"46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587\": rpc error: code = NotFound desc = could not find container \"46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587\": container with ID starting with 46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587 not found: ID does not exist"
Jul 09 18:28:21.222575 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.222532    2566 scope.go:117] "RemoveContainer" containerID="fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433"
Jul 09 18:28:21.222768 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:21.222750    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433\": container with ID starting with fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433 not found: ID does not exist" containerID="fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433"
Jul 09 18:28:21.222837 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.222775    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433"} err="failed to get container status \"fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433\": rpc error: code = NotFound desc = could not find container \"fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433\": container with ID starting with fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433 not found: ID does not exist"
Jul 09 18:28:21.222837 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.222793    2566 scope.go:117] "RemoveContainer" containerID="aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca"
Jul 09 18:28:21.223027 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:21.223010    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca\": container with ID starting with aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca not found: ID does not exist" containerID="aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca"
Jul 09 18:28:21.223072 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.223032    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca"} err="failed to get container status \"aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca\": rpc error: code = NotFound desc = could not find container \"aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca\": container with ID starting with aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca not found: ID does not exist"
Jul 09 18:28:21.223072 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.223046    2566 scope.go:117] "RemoveContainer" containerID="754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14"
Jul 09 18:28:21.223280 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:21.223263    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14\": container with ID starting with 754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14 not found: ID does not exist" containerID="754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14"
Jul 09 18:28:21.223319 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.223283    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14"} err="failed to get container status \"754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14\": rpc error: code = NotFound desc = could not find container \"754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14\": container with ID starting with 754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14 not found: ID does not exist"
Jul 09 18:28:21.223319 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.223297    2566 scope.go:117] "RemoveContainer" containerID="ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628"
Jul 09 18:28:21.223502 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:21.223488    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628\": container with ID starting with ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628 not found: ID does not exist" containerID="ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628"
Jul 09 18:28:21.223543 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.223504    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628"} err="failed to get container status \"ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628\": rpc error: code = NotFound desc = could not find container \"ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628\": container with ID starting with ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628 not found: ID does not exist"
Jul 09 18:28:21.223543 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.223515    2566 scope.go:117] "RemoveContainer" containerID="797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563"
Jul 09 18:28:21.223724 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:21.223708    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563\": container with ID starting with 797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563 not found: ID does not exist" containerID="797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563"
Jul 09 18:28:21.223767 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.223730    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563"} err="failed to get container status \"797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563\": rpc error: code = NotFound desc = could not find container \"797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563\": container with ID starting with 797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563 not found: ID does not exist"
Jul 09 18:28:21.223767 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.223744    2566 scope.go:117] "RemoveContainer" containerID="a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c"
Jul 09 18:28:21.223949 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.223930    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c"} err="failed to get container status \"a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c\": rpc error: code = NotFound desc = could not find container \"a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c\": container with ID starting with a23ef4bcffaac364d5902ed04f3fbf87d489a4f1eb3aa6e659d09051569cf69c not found: ID does not exist"
Jul 09 18:28:21.223993 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.223950    2566 scope.go:117] "RemoveContainer" containerID="46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587"
Jul 09 18:28:21.224176 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.224159    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587"} err="failed to get container status \"46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587\": rpc error: code = NotFound desc = could not find container \"46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587\": container with ID starting with 46b4ab811e80b627a24f7d5efd66fab63ccde0912acfe1f0ee52b951b30f5587 not found: ID does not exist"
Jul 09 18:28:21.224221 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.224175    2566 scope.go:117] "RemoveContainer" containerID="fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433"
Jul 09 18:28:21.224361 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.224347    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433"} err="failed to get container status \"fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433\": rpc error: code = NotFound desc = could not find container \"fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433\": container with ID starting with fbd95f811e97cc0e44575fd58229d0069981a3d7dc88ae0b684fe8a6ae7df433 not found: ID does not exist"
Jul 09 18:28:21.224398 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.224362    2566 scope.go:117] "RemoveContainer" containerID="aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca"
Jul 09 18:28:21.224562 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.224548    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca"} err="failed to get container status \"aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca\": rpc error: code = NotFound desc = could not find container \"aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca\": container with ID starting with aec12c9644194952c0e05be9e608c23677b150dbfd16d43f71af312d3db9b4ca not found: ID does not exist"
Jul 09 18:28:21.224562 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.224561    2566 scope.go:117] "RemoveContainer" containerID="754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14"
Jul 09 18:28:21.224783 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.224764    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14"} err="failed to get container status \"754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14\": rpc error: code = NotFound desc = could not find container \"754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14\": container with ID starting with 754bf45c7ce7d40e60474cb2eeb1b1c2ebe695209572fe9bd675ce36c1553e14 not found: ID does not exist"
Jul 09 18:28:21.224827 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.224784    2566 scope.go:117] "RemoveContainer" containerID="ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628"
Jul 09 18:28:21.224990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.224971    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628"} err="failed to get container status \"ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628\": rpc error: code = NotFound desc = could not find container \"ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628\": container with ID starting with ce82d17cd13040379593c71fdf82971e02201cdc65d369f42554f78d7dfc1628 not found: ID does not exist"
Jul 09 18:28:21.225042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.224990    2566 scope.go:117] "RemoveContainer" containerID="797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563"
Jul 09 18:28:21.225194 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.225179    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563"} err="failed to get container status \"797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563\": rpc error: code = NotFound desc = could not find container \"797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563\": container with ID starting with 797d088be8efb23a8758ea4e94137e56e8b2ea363ca4a6a503ccd9bf441b1563 not found: ID does not exist"
Jul 09 18:28:21.275286 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275262    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:28:21.275581 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275569    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy-web"
Jul 09 18:28:21.275621 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275583    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy-web"
Jul 09 18:28:21.275621 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275595    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32c563e6-5352-453d-b256-e3ed5819a18c" containerName="registry"
Jul 09 18:28:21.275621 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275601    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="32c563e6-5352-453d-b256-e3ed5819a18c" containerName="registry"
Jul 09 18:28:21.275621 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275608    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59557c96-fd65-44e1-8d99-a42a2dac5632" containerName="console"
Jul 09 18:28:21.275621 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275613    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="59557c96-fd65-44e1-8d99-a42a2dac5632" containerName="console"
Jul 09 18:28:21.275621 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275620    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="init-config-reloader"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275627    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="init-config-reloader"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275641    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="config-reloader"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275659    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="config-reloader"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275665    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy-metric"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275671    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy-metric"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275677    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275682    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275688    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" containerName="console"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275693    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" containerName="console"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275702    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="alertmanager"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275709    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="alertmanager"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275718    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="prom-label-proxy"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275725    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="prom-label-proxy"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275768    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="config-reloader"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275778    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="32c563e6-5352-453d-b256-e3ed5819a18c" containerName="registry"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275782    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d569a641-bc3e-4721-bc1f-e6b5bfd1f60c" containerName="console"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275789    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy-metric"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275801    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275809    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="prom-label-proxy"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275816    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="kube-rbac-proxy-web"
Jul 09 18:28:21.275813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275821    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="59557c96-fd65-44e1-8d99-a42a2dac5632" containerName="console"
Jul 09 18:28:21.276413 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.275826    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" containerName="alertmanager"
Jul 09 18:28:21.279346 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.279330    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.290393 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.290378    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 18:28:21.290776 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.290758    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 18:28:21.290860 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.290814    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 18:28:21.290982 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.290883    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sk6tg\""
Jul 09 18:28:21.291129 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.291117    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 18:28:21.292635 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.292621    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 18:28:21.293302 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.293286    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 18:28:21.293366 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.293304    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 18:28:21.293366 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.293360    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 18:28:21.298078 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.298063    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 18:28:21.305841 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.305822    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/593b26cd-bff8-426a-9d28-0cee362766b8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.305891 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.305847    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/593b26cd-bff8-426a-9d28-0cee362766b8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.305891 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.305874    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/593b26cd-bff8-426a-9d28-0cee362766b8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.305969 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.305903    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-web-config\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.305969 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.305935    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-config-volume\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.305969 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.305951    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.306078 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.305975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.306078 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.306025    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.306078 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.306071    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.306183 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.306101    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/593b26cd-bff8-426a-9d28-0cee362766b8-config-out\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.306183 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.306131    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cphw\" (UniqueName: \"kubernetes.io/projected/593b26cd-bff8-426a-9d28-0cee362766b8-kube-api-access-7cphw\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.306241 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.306191    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.306241 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.306231    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/593b26cd-bff8-426a-9d28-0cee362766b8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.320704 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.320678    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:28:21.357914 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.357890    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da92cf30-e6db-419b-9b62-ccdadf71f2ae" path="/var/lib/kubelet/pods/da92cf30-e6db-419b-9b62-ccdadf71f2ae/volumes"
Jul 09 18:28:21.406856 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.406836    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.406949 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.406863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.406949 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.406881    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.406949 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.406901    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.407046 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.407019    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/593b26cd-bff8-426a-9d28-0cee362766b8-config-out\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.407077 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.407049    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cphw\" (UniqueName: \"kubernetes.io/projected/593b26cd-bff8-426a-9d28-0cee362766b8-kube-api-access-7cphw\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.407212 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.407189    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.407287 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.407259    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/593b26cd-bff8-426a-9d28-0cee362766b8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.407346 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.407293    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/593b26cd-bff8-426a-9d28-0cee362766b8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.407346 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.407322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/593b26cd-bff8-426a-9d28-0cee362766b8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.407451 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.407363    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/593b26cd-bff8-426a-9d28-0cee362766b8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.407451 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.407406    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-web-config\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.407451 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.407432    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-config-volume\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.409255 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.408197    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/593b26cd-bff8-426a-9d28-0cee362766b8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.409255 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.408589    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/593b26cd-bff8-426a-9d28-0cee362766b8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.409255 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.409188    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/593b26cd-bff8-426a-9d28-0cee362766b8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.410227 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.410183    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-config-volume\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.410329 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.410241    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/593b26cd-bff8-426a-9d28-0cee362766b8-config-out\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.410329 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.410240    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.410431 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.410343    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.410511 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.410488    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.410672 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.410628    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.411021 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.411003    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.411430 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.411414    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/593b26cd-bff8-426a-9d28-0cee362766b8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.411585 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.411570    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/593b26cd-bff8-426a-9d28-0cee362766b8-web-config\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.425932 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.425879    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cphw\" (UniqueName: \"kubernetes.io/projected/593b26cd-bff8-426a-9d28-0cee362766b8-kube-api-access-7cphw\") pod \"alertmanager-main-0\" (UID: \"593b26cd-bff8-426a-9d28-0cee362766b8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.588083 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.588060    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:28:21.740383 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:21.740354    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:28:21.742637 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:28:21.742612    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod593b26cd_bff8_426a_9d28_0cee362766b8.slice/crio-aaa350c653a2805412a6fe06be500ffa5ca9b30942f4934b02faf7504d47fd20 WatchSource:0}: Error finding container aaa350c653a2805412a6fe06be500ffa5ca9b30942f4934b02faf7504d47fd20: Status 404 returned error can't find the container with id aaa350c653a2805412a6fe06be500ffa5ca9b30942f4934b02faf7504d47fd20
Jul 09 18:28:22.174331 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:22.174302    2566 generic.go:356] "Generic (PLEG): container finished" podID="593b26cd-bff8-426a-9d28-0cee362766b8" containerID="7ccad1162ead61dbb897d882ef88dbef89f342ed343bef664df9f2095b34415d" exitCode=0
Jul 09 18:28:22.174753 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:22.174354    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"593b26cd-bff8-426a-9d28-0cee362766b8","Type":"ContainerDied","Data":"7ccad1162ead61dbb897d882ef88dbef89f342ed343bef664df9f2095b34415d"}
Jul 09 18:28:22.174753 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:22.174432    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"593b26cd-bff8-426a-9d28-0cee362766b8","Type":"ContainerStarted","Data":"aaa350c653a2805412a6fe06be500ffa5ca9b30942f4934b02faf7504d47fd20"}
Jul 09 18:28:23.181993 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.181961    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"593b26cd-bff8-426a-9d28-0cee362766b8","Type":"ContainerStarted","Data":"114be8883af2d049a62dec2e4a908770eb45ce585c8999f915dcad50fca41685"}
Jul 09 18:28:23.182353 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.182046    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"593b26cd-bff8-426a-9d28-0cee362766b8","Type":"ContainerStarted","Data":"b73c8ad558b41461e72584af6eb695b1517df0b44ae8ba08f7023fb0d30924ca"}
Jul 09 18:28:23.182353 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.182076    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"593b26cd-bff8-426a-9d28-0cee362766b8","Type":"ContainerStarted","Data":"72c94ba08486955b15bf2df13c271d92b758225313878556d19b8b59a33531a3"}
Jul 09 18:28:23.182353 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.182102    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"593b26cd-bff8-426a-9d28-0cee362766b8","Type":"ContainerStarted","Data":"9256d60bc44d9f6c2deb6d527247dab7da0e50e112286c37fad7790894b2fbff"}
Jul 09 18:28:23.182353 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.182130    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"593b26cd-bff8-426a-9d28-0cee362766b8","Type":"ContainerStarted","Data":"b9d73b5c87427a7cf61adc6fe55aab34a8435ba952d2044db730833c068b643d"}
Jul 09 18:28:23.182353 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.182156    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"593b26cd-bff8-426a-9d28-0cee362766b8","Type":"ContainerStarted","Data":"b3626dbd326f54153f4b77e687dcdd92314aa1f1f6098435a90c9a7b9d8f9ec6"}
Jul 09 18:28:23.232503 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.232447    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.217632452 podStartE2EDuration="2.232427782s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.79533ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:28:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:28:23.229089289 +0000 UTC m=+180.364906762" watchObservedRunningTime="2026-07-09 18:28:23.232427782 +0000 UTC m=+180.368245256"
Jul 09 18:28:23.801313 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.801283    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:28:23.801603 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.801557    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="prometheus" containerID="cri-o://e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94" gracePeriod=600
Jul 09 18:28:23.801772 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.801601    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy" containerID="cri-o://c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce" gracePeriod=600
Jul 09 18:28:23.801772 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.801593    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="config-reloader" containerID="cri-o://299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62" gracePeriod=600
Jul 09 18:28:23.801772 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.801579    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="thanos-sidecar" containerID="cri-o://d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586" gracePeriod=600
Jul 09 18:28:23.801772 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.801629    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy-web" containerID="cri-o://ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad" gracePeriod=600
Jul 09 18:28:23.801772 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:23.801596    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea" gracePeriod=600
Jul 09 18:28:24.188772 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:24.188749    2566 generic.go:356] "Generic (PLEG): container finished" podID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerID="2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea" exitCode=0
Jul 09 18:28:24.188772 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:24.188770    2566 generic.go:356] "Generic (PLEG): container finished" podID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerID="c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce" exitCode=0
Jul 09 18:28:24.189086 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:24.188779    2566 generic.go:356] "Generic (PLEG): container finished" podID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerID="d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586" exitCode=0
Jul 09 18:28:24.189086 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:24.188785    2566 generic.go:356] "Generic (PLEG): container finished" podID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerID="299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62" exitCode=0
Jul 09 18:28:24.189086 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:24.188790    2566 generic.go:356] "Generic (PLEG): container finished" podID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerID="e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94" exitCode=0
Jul 09 18:28:24.189086 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:24.188834    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerDied","Data":"2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea"}
Jul 09 18:28:24.189086 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:24.188898    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerDied","Data":"c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce"}
Jul 09 18:28:24.189086 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:24.188930    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerDied","Data":"d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586"}
Jul 09 18:28:24.189086 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:24.188954    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerDied","Data":"299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62"}
Jul 09 18:28:24.189086 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:24.188982    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerDied","Data":"e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94"}
Jul 09 18:28:25.044380 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.044359    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.138293 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138223    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-1\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138293 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138255    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgvv7\" (UniqueName: \"kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-kube-api-access-cgvv7\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138293 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138273    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-grpc-tls\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138293 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138291    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-0\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138581 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138309    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-tls\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138581 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138388    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138581 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138431    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-tls-assets\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138581 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138459    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-kube-rbac-proxy\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138581 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138493    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138581 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138524    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-metrics-client-ca\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138581 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138561    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-2\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138996 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138602    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-thanos-prometheus-http-client-file\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138996 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138630    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-trusted-ca-bundle\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138996 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138690    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-db\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138996 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138722    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-web-config\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138996 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138758    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-kubelet-serving-ca-bundle\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138996 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138788    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config-out\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138996 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138810    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138996 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138849    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-serving-certs-ca-bundle\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.138996 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138889    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-metrics-client-certs\") pod \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\" (UID: \"a8171757-5f32-4b76-8e74-8dbdc43dcfcb\") "
Jul 09 18:28:25.139469 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.138624    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:25.139872 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.139528    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:25.139872 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.139747    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:25.140487 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.140454    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:25.140487 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.140465    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:25.140743 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.140719    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:25.141515 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.141487    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:28:25.141618 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.141493    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:25.141803 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.141780    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:28:25.142204 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.142175    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-kube-api-access-cgvv7" (OuterVolumeSpecName: "kube-api-access-cgvv7") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "kube-api-access-cgvv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:28:25.142378 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.142352    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config" (OuterVolumeSpecName: "config") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:25.142599 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.142577    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:28:25.142736 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.142718    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:25.143079 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.143043    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:25.143271 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.143246    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:25.143470 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.143433    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:25.143675 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.143630    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:25.144129 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.144112    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:25.144352 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.144331    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config-out" (OuterVolumeSpecName: "config-out") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:28:25.153284 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.153265    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-web-config" (OuterVolumeSpecName: "web-config") pod "a8171757-5f32-4b76-8e74-8dbdc43dcfcb" (UID: "a8171757-5f32-4b76-8e74-8dbdc43dcfcb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:25.195935 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.195913    2566 generic.go:356] "Generic (PLEG): container finished" podID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerID="ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad" exitCode=0
Jul 09 18:28:25.196197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.195979    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.196197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.196000    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerDied","Data":"ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad"}
Jul 09 18:28:25.196197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.196069    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8171757-5f32-4b76-8e74-8dbdc43dcfcb","Type":"ContainerDied","Data":"3e589e04c8e4f70ac0c005a756b8f467972c27dd128b5ba9399efbd659785a77"}
Jul 09 18:28:25.196197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.196104    2566 scope.go:117] "RemoveContainer" containerID="2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea"
Jul 09 18:28:25.203574 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.203553    2566 scope.go:117] "RemoveContainer" containerID="c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce"
Jul 09 18:28:25.210100 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.210083    2566 scope.go:117] "RemoveContainer" containerID="ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad"
Jul 09 18:28:25.216560 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.216541    2566 scope.go:117] "RemoveContainer" containerID="d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586"
Jul 09 18:28:25.221394 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.221371    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:28:25.223008 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.222992    2566 scope.go:117] "RemoveContainer" containerID="299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62"
Jul 09 18:28:25.227352 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.227331    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:28:25.229890 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.229875    2566 scope.go:117] "RemoveContainer" containerID="e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94"
Jul 09 18:28:25.236564 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.236544    2566 scope.go:117] "RemoveContainer" containerID="62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332"
Jul 09 18:28:25.239718 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239701    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.239785 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239720    2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.239785 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239732    2566 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.239785 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239740    2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-tls-assets\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.239785 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239748    2566 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-kube-rbac-proxy\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.239785 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239758    2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.239785 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239767    2566 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-metrics-client-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.239785 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239776    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.239785 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239784    2566 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239793    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239802    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-db\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239809    2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-web-config\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239817    2566 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239826    2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-config-out\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239834    2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239841    2566 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239849    2566 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-metrics-client-certs\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239857    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239865    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgvv7\" (UniqueName: \"kubernetes.io/projected/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-kube-api-access-cgvv7\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.240062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.239874    2566 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8171757-5f32-4b76-8e74-8dbdc43dcfcb-secret-grpc-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:28:25.242815 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.242799    2566 scope.go:117] "RemoveContainer" containerID="2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea"
Jul 09 18:28:25.243062 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:25.243044    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea\": container with ID starting with 2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea not found: ID does not exist" containerID="2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea"
Jul 09 18:28:25.243105 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.243071    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea"} err="failed to get container status \"2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea\": rpc error: code = NotFound desc = could not find container \"2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea\": container with ID starting with 2ef4aa7c274a1c781e3d1b02107b4032d7a345f5fdaf9e350cc5cf407510c1ea not found: ID does not exist"
Jul 09 18:28:25.243105 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.243089    2566 scope.go:117] "RemoveContainer" containerID="c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce"
Jul 09 18:28:25.243334 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:25.243319    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce\": container with ID starting with c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce not found: ID does not exist" containerID="c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce"
Jul 09 18:28:25.243380 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.243338    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce"} err="failed to get container status \"c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce\": rpc error: code = NotFound desc = could not find container \"c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce\": container with ID starting with c6d7c93c1162cd2fc81cbc56d9a4364b01855bc8a663cbcf011b0a8a264d10ce not found: ID does not exist"
Jul 09 18:28:25.243380 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.243350    2566 scope.go:117] "RemoveContainer" containerID="ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad"
Jul 09 18:28:25.243548 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:25.243532    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad\": container with ID starting with ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad not found: ID does not exist" containerID="ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad"
Jul 09 18:28:25.243589 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.243554    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad"} err="failed to get container status \"ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad\": rpc error: code = NotFound desc = could not find container \"ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad\": container with ID starting with ef31c4f8ca7b53697f72ae36b4c6cbb98df3340f8c356fe5121280420a0e59ad not found: ID does not exist"
Jul 09 18:28:25.243589 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.243570    2566 scope.go:117] "RemoveContainer" containerID="d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586"
Jul 09 18:28:25.243770 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:25.243754    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586\": container with ID starting with d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586 not found: ID does not exist" containerID="d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586"
Jul 09 18:28:25.243804 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.243773    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586"} err="failed to get container status \"d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586\": rpc error: code = NotFound desc = could not find container \"d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586\": container with ID starting with d79a534fc8271847fd6c7365fd70b291c4b9686055e646d716b78ee828c14586 not found: ID does not exist"
Jul 09 18:28:25.243804 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.243797    2566 scope.go:117] "RemoveContainer" containerID="299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62"
Jul 09 18:28:25.243979 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:25.243964    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62\": container with ID starting with 299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62 not found: ID does not exist" containerID="299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62"
Jul 09 18:28:25.244021 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.243982    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62"} err="failed to get container status \"299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62\": rpc error: code = NotFound desc = could not find container \"299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62\": container with ID starting with 299bd1e925976edd676daeda34b217142dca4c00c08b3eb470138d826ee5ab62 not found: ID does not exist"
Jul 09 18:28:25.244021 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.243993    2566 scope.go:117] "RemoveContainer" containerID="e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94"
Jul 09 18:28:25.244168 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:25.244152    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94\": container with ID starting with e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94 not found: ID does not exist" containerID="e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94"
Jul 09 18:28:25.244206 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.244171    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94"} err="failed to get container status \"e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94\": rpc error: code = NotFound desc = could not find container \"e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94\": container with ID starting with e0af52d4f3c6e057148ca431fe73d12a4984f1f2dba9f08b808626283be7af94 not found: ID does not exist"
Jul 09 18:28:25.244206 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.244184    2566 scope.go:117] "RemoveContainer" containerID="62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332"
Jul 09 18:28:25.244357 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:28:25.244338    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332\": container with ID starting with 62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332 not found: ID does not exist" containerID="62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332"
Jul 09 18:28:25.244394 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.244360    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332"} err="failed to get container status \"62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332\": rpc error: code = NotFound desc = could not find container \"62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332\": container with ID starting with 62799b7eb5a396f52a2297679b747f6bed51b8a8dc4e5e009626137efda47332 not found: ID does not exist"
Jul 09 18:28:25.259099 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259076    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:28:25.259344 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259332    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy-thanos"
Jul 09 18:28:25.259391 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259347    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy-thanos"
Jul 09 18:28:25.259391 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259358    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="thanos-sidecar"
Jul 09 18:28:25.259391 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259363    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="thanos-sidecar"
Jul 09 18:28:25.259391 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259372    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="prometheus"
Jul 09 18:28:25.259391 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259377    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="prometheus"
Jul 09 18:28:25.259391 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259390    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy-web"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259395    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy-web"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259402    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="init-config-reloader"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259407    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="init-config-reloader"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259414    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259419    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259426    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="config-reloader"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259431    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="config-reloader"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259469    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="config-reloader"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259477    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy-thanos"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259484    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="thanos-sidecar"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259489    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy-web"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259496    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="prometheus"
Jul 09 18:28:25.259557 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.259502    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" containerName="kube-rbac-proxy"
Jul 09 18:28:25.264872 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.264094    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.268291 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.268261    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6pqfg\""
Jul 09 18:28:25.268522 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.268504    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 18:28:25.269434 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.269412    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 18:28:25.269810 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.269727    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 18:28:25.269810 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.269743    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 18:28:25.269810 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.269749    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 18:28:25.269810 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.269759    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 18:28:25.270063 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.269914    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1n95bka1a46mu\""
Jul 09 18:28:25.270274 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.270254    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 18:28:25.270366 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.270341    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 18:28:25.270448 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.270417    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 18:28:25.270712 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.270693    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 18:28:25.270812 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.270732    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 18:28:25.270867 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.270849    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 18:28:25.278513 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.278497    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 18:28:25.278803 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.278787    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 18:28:25.282087 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.282070    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:28:25.283685 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.283626    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 18:28:25.341036 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341015    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70b3e53a-50ef-4766-a5df-929d0c0b0620-config-out\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341118 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341041    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341118 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341056    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-web-config\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341118 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341075    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341118 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341100    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341279 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341138    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrz8q\" (UniqueName: \"kubernetes.io/projected/70b3e53a-50ef-4766-a5df-929d0c0b0620-kube-api-access-zrz8q\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341279 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341176    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341279 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341195    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341279 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341212    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341279 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341230    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341279 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341247    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341516 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341303    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341516 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341343    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-config\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341516 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341374    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70b3e53a-50ef-4766-a5df-929d0c0b0620-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341516 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341439    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341516 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341463    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341516 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341483    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341750 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341518    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341750 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341545    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.341750 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.341562    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.359762 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.359736    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8171757-5f32-4b76-8e74-8dbdc43dcfcb" path="/var/lib/kubelet/pods/a8171757-5f32-4b76-8e74-8dbdc43dcfcb/volumes"
Jul 09 18:28:25.442487 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.442465    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.442595 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.442496    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70b3e53a-50ef-4766-a5df-929d0c0b0620-config-out\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.442595 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.442521    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.442595 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.442545    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-web-config\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.442745 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.442679    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.442745 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.442718    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.442880 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.442858    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrz8q\" (UniqueName: \"kubernetes.io/projected/70b3e53a-50ef-4766-a5df-929d0c0b0620-kube-api-access-zrz8q\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.442945 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.442912    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.442997 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.442941    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.442997 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.442976    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443097 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443008    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443097 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443040    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443097 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443074    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443245 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443110    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-config\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443245 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443138    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70b3e53a-50ef-4766-a5df-929d0c0b0620-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443245 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443180    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443245 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443245 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443239    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443483 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443256    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443483 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443300    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443483 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443334    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.443816 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.443792    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.444330 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.444303    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.444426 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.444411    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.446347 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.446318    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.447009 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.446713    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.447009 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.446968    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.447623 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.447387    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.447623 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.447434    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.447623 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.447437    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.447623 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.447584    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.447870 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.447640    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.447870 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.447690    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70b3e53a-50ef-4766-a5df-929d0c0b0620-config-out\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.447966 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.447899    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70b3e53a-50ef-4766-a5df-929d0c0b0620-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.448018 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.447984    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.448018 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.448001    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-web-config\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.448156 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.448129    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-config\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.448344 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.448326    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/70b3e53a-50ef-4766-a5df-929d0c0b0620-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.448809 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.448791    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/70b3e53a-50ef-4766-a5df-929d0c0b0620-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.454738 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.454718    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrz8q\" (UniqueName: \"kubernetes.io/projected/70b3e53a-50ef-4766-a5df-929d0c0b0620-kube-api-access-zrz8q\") pod \"prometheus-k8s-0\" (UID: \"70b3e53a-50ef-4766-a5df-929d0c0b0620\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.575580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.575555    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:28:25.736995 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:25.736905    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:28:25.741092 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:28:25.741055    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70b3e53a_50ef_4766_a5df_929d0c0b0620.slice/crio-df2d2462d4a3b83127e208fe0c4edc2827b09b7275d04585ffc792da6f79b3c4 WatchSource:0}: Error finding container df2d2462d4a3b83127e208fe0c4edc2827b09b7275d04585ffc792da6f79b3c4: Status 404 returned error can't find the container with id df2d2462d4a3b83127e208fe0c4edc2827b09b7275d04585ffc792da6f79b3c4
Jul 09 18:28:26.201571 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:26.201543    2566 generic.go:356] "Generic (PLEG): container finished" podID="70b3e53a-50ef-4766-a5df-929d0c0b0620" containerID="28314aa6d819c2eae965f839dad3eef58697111903b915159ac3c3f1a89d7904" exitCode=0
Jul 09 18:28:26.201983 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:26.201628    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70b3e53a-50ef-4766-a5df-929d0c0b0620","Type":"ContainerDied","Data":"28314aa6d819c2eae965f839dad3eef58697111903b915159ac3c3f1a89d7904"}
Jul 09 18:28:26.201983 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:26.201727    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70b3e53a-50ef-4766-a5df-929d0c0b0620","Type":"ContainerStarted","Data":"df2d2462d4a3b83127e208fe0c4edc2827b09b7275d04585ffc792da6f79b3c4"}
Jul 09 18:28:27.209362 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:27.209333    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70b3e53a-50ef-4766-a5df-929d0c0b0620","Type":"ContainerStarted","Data":"ff8b27ef22ee5be870a2aaa7e3bb69eb5d3ab15ce3f201e20e2bed7fa6274576"}
Jul 09 18:28:27.209733 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:27.209396    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70b3e53a-50ef-4766-a5df-929d0c0b0620","Type":"ContainerStarted","Data":"04907218ca8b862ca04bcb49c0cd683c05e83caa641b4d05249788850a45c403"}
Jul 09 18:28:27.209733 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:27.209422    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70b3e53a-50ef-4766-a5df-929d0c0b0620","Type":"ContainerStarted","Data":"992610fdccbd8010f770de3f7094e54a3747088be6c5b9cb17a1fa4dc5c9aa71"}
Jul 09 18:28:27.209733 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:27.209453    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70b3e53a-50ef-4766-a5df-929d0c0b0620","Type":"ContainerStarted","Data":"e891cf951d69ce6c818687223a693011fd308dc8c6569b568aef46904d827701"}
Jul 09 18:28:27.209733 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:27.209478    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70b3e53a-50ef-4766-a5df-929d0c0b0620","Type":"ContainerStarted","Data":"4bfc89a4aad6f1cd945a7c9bca43aa9dc6b3095f8119a71f00706b0ab99ad0e7"}
Jul 09 18:28:27.209733 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:27.209502    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70b3e53a-50ef-4766-a5df-929d0c0b0620","Type":"ContainerStarted","Data":"f6009a488f89242c6f61cf6c5157906664b82c5c740c95a9adaa03180b64e163"}
Jul 09 18:28:27.250344 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:27.250294    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.227364331 podStartE2EDuration="2.250275611s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.91128ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:28:27.249518443 +0000 UTC m=+184.385335911" watchObservedRunningTime="2026-07-09 18:28:27.250275611 +0000 UTC m=+184.386093084"
Jul 09 18:28:30.576412 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:28:30.576378    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:29:25.576148 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:25.576110    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:29:25.591736 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:25.591713    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:29:26.455518 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:26.455489    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:29:30.994573 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:30.994531    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"]
Jul 09 18:29:30.999306 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:30.999286    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"
Jul 09 18:29:31.001859 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.001827    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 18:29:31.001998 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.001830    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-gdp4j\""
Jul 09 18:29:31.002794 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.002772    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 18:29:31.009367 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.009339    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"]
Jul 09 18:29:31.021262 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.021239    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccc7917f-5571-4413-9f76-92d981fbc82b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cgxwc\" (UID: \"ccc7917f-5571-4413-9f76-92d981fbc82b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"
Jul 09 18:29:31.021362 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.021306    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42j9v\" (UniqueName: \"kubernetes.io/projected/ccc7917f-5571-4413-9f76-92d981fbc82b-kube-api-access-42j9v\") pod \"cert-manager-cainjector-559cdd8c87-cgxwc\" (UID: \"ccc7917f-5571-4413-9f76-92d981fbc82b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"
Jul 09 18:29:31.121804 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.121780    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42j9v\" (UniqueName: \"kubernetes.io/projected/ccc7917f-5571-4413-9f76-92d981fbc82b-kube-api-access-42j9v\") pod \"cert-manager-cainjector-559cdd8c87-cgxwc\" (UID: \"ccc7917f-5571-4413-9f76-92d981fbc82b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"
Jul 09 18:29:31.121895 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.121824    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccc7917f-5571-4413-9f76-92d981fbc82b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cgxwc\" (UID: \"ccc7917f-5571-4413-9f76-92d981fbc82b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"
Jul 09 18:29:31.131980 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.131950    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccc7917f-5571-4413-9f76-92d981fbc82b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cgxwc\" (UID: \"ccc7917f-5571-4413-9f76-92d981fbc82b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"
Jul 09 18:29:31.132145 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.132126    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42j9v\" (UniqueName: \"kubernetes.io/projected/ccc7917f-5571-4413-9f76-92d981fbc82b-kube-api-access-42j9v\") pod \"cert-manager-cainjector-559cdd8c87-cgxwc\" (UID: \"ccc7917f-5571-4413-9f76-92d981fbc82b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"
Jul 09 18:29:31.325740 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.325677    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"
Jul 09 18:29:31.455139 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.455118    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc"]
Jul 09 18:29:31.457569 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:29:31.457541    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podccc7917f_5571_4413_9f76_92d981fbc82b.slice/crio-7eb08db0b7379f4bc75dac04453531190292188a669f70f9902d8c90c1b65165 WatchSource:0}: Error finding container 7eb08db0b7379f4bc75dac04453531190292188a669f70f9902d8c90c1b65165: Status 404 returned error can't find the container with id 7eb08db0b7379f4bc75dac04453531190292188a669f70f9902d8c90c1b65165
Jul 09 18:29:31.466263 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:31.466241    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc" event={"ID":"ccc7917f-5571-4413-9f76-92d981fbc82b","Type":"ContainerStarted","Data":"7eb08db0b7379f4bc75dac04453531190292188a669f70f9902d8c90c1b65165"}
Jul 09 18:29:34.480789 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:34.480760    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc" event={"ID":"ccc7917f-5571-4413-9f76-92d981fbc82b","Type":"ContainerStarted","Data":"bcd767d333ef10256064f8863e24a6e66e590852961202059d7aca030f637e7a"}
Jul 09 18:29:34.499056 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:29:34.499011    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cgxwc" podStartSLOduration=1.579423051 podStartE2EDuration="4.498996589s" totalImagesPullingTime="2.919573538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:29:34.497997916 +0000 UTC m=+251.633815391" watchObservedRunningTime="2026-07-09 18:29:34.498996589 +0000 UTC m=+251.634814061"
Jul 09 18:30:15.836654 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.836600    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"]
Jul 09 18:30:15.840082 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.840063    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:15.845468 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.845416    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fmrpn\""
Jul 09 18:30:15.845589 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.845532    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 18:30:15.845589 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.845550    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 18:30:15.845788 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.845770    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 18:30:15.846007 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.845991    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 18:30:15.858196 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.858171    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"]
Jul 09 18:30:15.937747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.937726    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzzvt\" (UniqueName: \"kubernetes.io/projected/08d0246d-eac4-4424-b007-6ee9d2a34f1c-kube-api-access-qzzvt\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:15.937848 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.937777    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08d0246d-eac4-4424-b007-6ee9d2a34f1c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:15.937905 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.937852    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08d0246d-eac4-4424-b007-6ee9d2a34f1c-tmp\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:15.937952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.937908    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08d0246d-eac4-4424-b007-6ee9d2a34f1c-manifests\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:15.937952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:15.937928    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08d0246d-eac4-4424-b007-6ee9d2a34f1c-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.038682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.038637    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08d0246d-eac4-4424-b007-6ee9d2a34f1c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.038797 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.038695    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08d0246d-eac4-4424-b007-6ee9d2a34f1c-tmp\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.038797 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.038719    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08d0246d-eac4-4424-b007-6ee9d2a34f1c-manifests\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.038797 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.038735    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08d0246d-eac4-4424-b007-6ee9d2a34f1c-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.038797 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.038772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzzvt\" (UniqueName: \"kubernetes.io/projected/08d0246d-eac4-4424-b007-6ee9d2a34f1c-kube-api-access-qzzvt\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.039177 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.039154    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08d0246d-eac4-4424-b007-6ee9d2a34f1c-manifests\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.041093 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.041071    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08d0246d-eac4-4424-b007-6ee9d2a34f1c-tmp\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.041209 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.041189    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08d0246d-eac4-4424-b007-6ee9d2a34f1c-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.041400 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.041382    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08d0246d-eac4-4424-b007-6ee9d2a34f1c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.067724 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.067699    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzzvt\" (UniqueName: \"kubernetes.io/projected/08d0246d-eac4-4424-b007-6ee9d2a34f1c-kube-api-access-qzzvt\") pod \"opendatahub-operator-controller-manager-59f7885b79-hv86z\" (UID: \"08d0246d-eac4-4424-b007-6ee9d2a34f1c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.149940 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.149888    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:16.288107 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.288080    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"]
Jul 09 18:30:16.290807 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:30:16.290779    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d0246d_eac4_4424_b007_6ee9d2a34f1c.slice/crio-77c3cf405eabb7eef869b96a261804234af068ff726fe183877b70812cf344e7 WatchSource:0}: Error finding container 77c3cf405eabb7eef869b96a261804234af068ff726fe183877b70812cf344e7: Status 404 returned error can't find the container with id 77c3cf405eabb7eef869b96a261804234af068ff726fe183877b70812cf344e7
Jul 09 18:30:16.647148 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:16.647119    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z" event={"ID":"08d0246d-eac4-4424-b007-6ee9d2a34f1c","Type":"ContainerStarted","Data":"77c3cf405eabb7eef869b96a261804234af068ff726fe183877b70812cf344e7"}
Jul 09 18:30:19.663452 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:19.663421    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z" event={"ID":"08d0246d-eac4-4424-b007-6ee9d2a34f1c","Type":"ContainerStarted","Data":"7fa0490d3fa8a1eb755b5bbb184e08975ceb1337843a526293fbbcd1c5c43429"}
Jul 09 18:30:20.669268 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:20.669235    2566 generic.go:356] "Generic (PLEG): container finished" podID="08d0246d-eac4-4424-b007-6ee9d2a34f1c" containerID="7fa0490d3fa8a1eb755b5bbb184e08975ceb1337843a526293fbbcd1c5c43429" exitCode=0
Jul 09 18:30:20.669575 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:20.669277    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z" event={"ID":"08d0246d-eac4-4424-b007-6ee9d2a34f1c","Type":"ContainerDied","Data":"7fa0490d3fa8a1eb755b5bbb184e08975ceb1337843a526293fbbcd1c5c43429"}
Jul 09 18:30:21.674945 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:21.674912    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z" event={"ID":"08d0246d-eac4-4424-b007-6ee9d2a34f1c","Type":"ContainerStarted","Data":"453f3451a16fe743f125fe120612d8bb48d0f8892d6607fb22dd94b00897d6fc"}
Jul 09 18:30:21.675318 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:21.675024    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:21.713843 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:21.713801    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z" podStartSLOduration=2.26757551 podStartE2EDuration="6.713788868s" totalImagesPullingTime="3.398496985s" totalInitContainerRuntime="1.047716373s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:30:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:30:21.710707161 +0000 UTC m=+298.846524632" watchObservedRunningTime="2026-07-09 18:30:21.713788868 +0000 UTC m=+298.849606339"
Jul 09 18:30:23.211268 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:23.211229    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 18:30:32.681413 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:32.681385    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59f7885b79-hv86z"
Jul 09 18:30:36.814863 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.814836    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"]
Jul 09 18:30:36.817235 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.817218    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.828577 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.828553    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 18:30:36.828836 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.828819    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 09 18:30:36.829066 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.829050    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-spnbx\""
Jul 09 18:30:36.829331 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.829308    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:30:36.829814 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.829781    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 09 18:30:36.831062 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.831043    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 09 18:30:36.836294 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.836272    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"]
Jul 09 18:30:36.887765 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.887745    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a10a622-b903-4d9a-9167-7c96e310e88b-metrics-cert\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.887847 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.887775    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptl6t\" (UniqueName: \"kubernetes.io/projected/0a10a622-b903-4d9a-9167-7c96e310e88b-kube-api-access-ptl6t\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.887847 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.887794    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0a10a622-b903-4d9a-9167-7c96e310e88b-manager-config\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.887929 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.887868    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a10a622-b903-4d9a-9167-7c96e310e88b-cert\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.988511 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.988478    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a10a622-b903-4d9a-9167-7c96e310e88b-metrics-cert\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.988702 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.988538    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptl6t\" (UniqueName: \"kubernetes.io/projected/0a10a622-b903-4d9a-9167-7c96e310e88b-kube-api-access-ptl6t\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.988702 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.988684    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0a10a622-b903-4d9a-9167-7c96e310e88b-manager-config\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.988823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.988742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a10a622-b903-4d9a-9167-7c96e310e88b-cert\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.989222 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.989197    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0a10a622-b903-4d9a-9167-7c96e310e88b-manager-config\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.990836 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.990814    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a10a622-b903-4d9a-9167-7c96e310e88b-metrics-cert\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:36.991093 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:36.991072    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a10a622-b903-4d9a-9167-7c96e310e88b-cert\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:37.004522 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:37.004486    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptl6t\" (UniqueName: \"kubernetes.io/projected/0a10a622-b903-4d9a-9167-7c96e310e88b-kube-api-access-ptl6t\") pod \"lws-controller-manager-576fd7db4f-mx7hr\" (UID: \"0a10a622-b903-4d9a-9167-7c96e310e88b\") " pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:37.127004 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:37.126939    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:37.284049 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:37.284020    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"]
Jul 09 18:30:37.285214 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:30:37.285186    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a10a622_b903_4d9a_9167_7c96e310e88b.slice/crio-ca1c7dccfa67cc0aac25a76b1ff638481fc6543c2f3008fc7998d444003ddb02 WatchSource:0}: Error finding container ca1c7dccfa67cc0aac25a76b1ff638481fc6543c2f3008fc7998d444003ddb02: Status 404 returned error can't find the container with id ca1c7dccfa67cc0aac25a76b1ff638481fc6543c2f3008fc7998d444003ddb02
Jul 09 18:30:37.287063 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:37.287046    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:30:37.747706 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:37.747667    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr" event={"ID":"0a10a622-b903-4d9a-9167-7c96e310e88b","Type":"ContainerStarted","Data":"ca1c7dccfa67cc0aac25a76b1ff638481fc6543c2f3008fc7998d444003ddb02"}
Jul 09 18:30:39.758108 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:39.758070    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr" event={"ID":"0a10a622-b903-4d9a-9167-7c96e310e88b","Type":"ContainerStarted","Data":"6164591a01764288c3530c0d82f150804d9c719d6f790226c0501d1bb6f312cd"}
Jul 09 18:30:39.758453 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:39.758141    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:30:39.796598 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:39.796559    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr" podStartSLOduration=1.806094881 podStartE2EDuration="3.796545576s" totalImagesPullingTime="1.990450695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:30:39.794858708 +0000 UTC m=+316.930676190" watchObservedRunningTime="2026-07-09 18:30:39.796545576 +0000 UTC m=+316.932363103"
Jul 09 18:30:43.636262 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.636227    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"]
Jul 09 18:30:43.638920 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.638903    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.642296 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.642268    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 09 18:30:43.642416 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.642385    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pqlc8\""
Jul 09 18:30:43.643068 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.643045    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 09 18:30:43.643068 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.643062    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 18:30:43.643250 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.643055    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 09 18:30:43.643250 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.643143    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 18:30:43.646526 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.646510    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 09 18:30:43.655010 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.654992    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"]
Jul 09 18:30:43.743339 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.743292    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.743447 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.743346    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.743447 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.743372    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.743514 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.743451    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pv2x\" (UniqueName: \"kubernetes.io/projected/7a966206-99c8-4a39-968f-e92a3d860cdd-kube-api-access-8pv2x\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.743514 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.743480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a966206-99c8-4a39-968f-e92a3d860cdd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.743514 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.743498    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.743608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.743558    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a966206-99c8-4a39-968f-e92a3d860cdd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.844544 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.844504    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pv2x\" (UniqueName: \"kubernetes.io/projected/7a966206-99c8-4a39-968f-e92a3d860cdd-kube-api-access-8pv2x\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.844696 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.844550    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a966206-99c8-4a39-968f-e92a3d860cdd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.844696 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.844566    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.844696 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.844684    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a966206-99c8-4a39-968f-e92a3d860cdd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.844834 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.844740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.844834 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.844770    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.844834 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.844795    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.845393 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.845368    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.846881 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.846855    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a966206-99c8-4a39-968f-e92a3d860cdd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.847073 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.847045    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.847256 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.847241    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a966206-99c8-4a39-968f-e92a3d860cdd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.847366 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.847348    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.854434 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.854408    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a966206-99c8-4a39-968f-e92a3d860cdd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.856504 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.856479    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pv2x\" (UniqueName: \"kubernetes.io/projected/7a966206-99c8-4a39-968f-e92a3d860cdd-kube-api-access-8pv2x\") pod \"istiod-openshift-gateway-94bb8fbfd-rn4db\" (UID: \"7a966206-99c8-4a39-968f-e92a3d860cdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:43.947853 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:43.947835    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:44.091839 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:44.091800    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"]
Jul 09 18:30:44.782249 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:44.782217    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db" event={"ID":"7a966206-99c8-4a39-968f-e92a3d860cdd","Type":"ContainerStarted","Data":"895f2d3f2c3f37e853182b8bad4483fc4f870c0bb022c704b9eb0318f9661846"}
Jul 09 18:30:46.089966 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:46.089928    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:30:46.090196 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:46.090000    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:30:46.795009 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:46.794979    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db" event={"ID":"7a966206-99c8-4a39-968f-e92a3d860cdd","Type":"ContainerStarted","Data":"de325b1e271fd9c36f12d62f1a73ced3abd6e02f55b96808bd0ef82ada92ea1e"}
Jul 09 18:30:46.795333 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:46.795113    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:46.842427 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:46.842382    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db" podStartSLOduration=1.852346 podStartE2EDuration="3.842367685s" totalImagesPullingTime="1.990021685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:30:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:30:46.836043612 +0000 UTC m=+323.971861093" watchObservedRunningTime="2026-07-09 18:30:46.842367685 +0000 UTC m=+323.978185155"
Jul 09 18:30:47.807445 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:47.807409    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rn4db"
Jul 09 18:30:50.763700 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:30:50.763639    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-576fd7db4f-mx7hr"
Jul 09 18:32:18.565199 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.565166    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"]
Jul 09 18:32:18.568426 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.568408    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:18.570867 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.570842    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 18:32:18.570980 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.570897    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 18:32:18.570980 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.570971    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r6xjq\""
Jul 09 18:32:18.587540 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.587512    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"]
Jul 09 18:32:18.671039 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.671002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f83c532f-2092-4dc3-ae7d-0eb899f1f852-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tt6pw\" (UID: \"f83c532f-2092-4dc3-ae7d-0eb899f1f852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:18.671216 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.671087    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kds6z\" (UniqueName: \"kubernetes.io/projected/f83c532f-2092-4dc3-ae7d-0eb899f1f852-kube-api-access-kds6z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tt6pw\" (UID: \"f83c532f-2092-4dc3-ae7d-0eb899f1f852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:18.772106 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.772071    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f83c532f-2092-4dc3-ae7d-0eb899f1f852-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tt6pw\" (UID: \"f83c532f-2092-4dc3-ae7d-0eb899f1f852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:18.772233 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.772155    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kds6z\" (UniqueName: \"kubernetes.io/projected/f83c532f-2092-4dc3-ae7d-0eb899f1f852-kube-api-access-kds6z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tt6pw\" (UID: \"f83c532f-2092-4dc3-ae7d-0eb899f1f852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:18.772510 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.772489    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f83c532f-2092-4dc3-ae7d-0eb899f1f852-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tt6pw\" (UID: \"f83c532f-2092-4dc3-ae7d-0eb899f1f852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:18.785205 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.785179    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kds6z\" (UniqueName: \"kubernetes.io/projected/f83c532f-2092-4dc3-ae7d-0eb899f1f852-kube-api-access-kds6z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tt6pw\" (UID: \"f83c532f-2092-4dc3-ae7d-0eb899f1f852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:18.878805 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:18.878726    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:19.019880 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:19.019851    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"]
Jul 09 18:32:19.021717 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:32:19.021688    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83c532f_2092_4dc3_ae7d_0eb899f1f852.slice/crio-d37ed446154c096f7ff5f27d0ef9372107bf8ffd9bbbd30335b20435ffaff399 WatchSource:0}: Error finding container d37ed446154c096f7ff5f27d0ef9372107bf8ffd9bbbd30335b20435ffaff399: Status 404 returned error can't find the container with id d37ed446154c096f7ff5f27d0ef9372107bf8ffd9bbbd30335b20435ffaff399
Jul 09 18:32:19.222773 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:19.222738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw" event={"ID":"f83c532f-2092-4dc3-ae7d-0eb899f1f852","Type":"ContainerStarted","Data":"d37ed446154c096f7ff5f27d0ef9372107bf8ffd9bbbd30335b20435ffaff399"}
Jul 09 18:32:28.267314 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:28.267276    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw" event={"ID":"f83c532f-2092-4dc3-ae7d-0eb899f1f852","Type":"ContainerStarted","Data":"4b107e4a0f543fcc2d44191a99e176416c846abdaa36a32a76df3b1080fd340d"}
Jul 09 18:32:28.267820 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:28.267412    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:28.295704 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:28.295639    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw" podStartSLOduration=2.021778058 podStartE2EDuration="10.295623782s" totalImagesPullingTime="8.273845724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:32:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:32:28.293770906 +0000 UTC m=+425.429588375" watchObservedRunningTime="2026-07-09 18:32:28.295623782 +0000 UTC m=+425.431441253"
Jul 09 18:32:39.274206 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:39.274170    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:41.189956 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.189916    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"]
Jul 09 18:32:41.190514 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.190137    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw" podUID="f83c532f-2092-4dc3-ae7d-0eb899f1f852" containerName="manager" containerID="cri-o://4b107e4a0f543fcc2d44191a99e176416c846abdaa36a32a76df3b1080fd340d" gracePeriod=2
Jul 09 18:32:41.192947 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:32:41.192908    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tt6pw\" is forbidden: User \"system:node:ip-10-0-135-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-28.ec2.internal' and this object" podUID="f83c532f-2092-4dc3-ae7d-0eb899f1f852" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:41.193247 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.193224    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"]
Jul 09 18:32:41.236357 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.236323    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"]
Jul 09 18:32:41.236734 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.236718    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83c532f-2092-4dc3-ae7d-0eb899f1f852" containerName="manager"
Jul 09 18:32:41.236824 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.236737    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83c532f-2092-4dc3-ae7d-0eb899f1f852" containerName="manager"
Jul 09 18:32:41.236887 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.236833    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83c532f-2092-4dc3-ae7d-0eb899f1f852" containerName="manager"
Jul 09 18:32:41.239794 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.239772    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:41.242280 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:32:41.242252    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tt6pw\" is forbidden: User \"system:node:ip-10-0-135-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-28.ec2.internal' and this object" podUID="f83c532f-2092-4dc3-ae7d-0eb899f1f852" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:41.263108 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.262871    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"]
Jul 09 18:32:41.325541 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.325508    2566 generic.go:356] "Generic (PLEG): container finished" podID="f83c532f-2092-4dc3-ae7d-0eb899f1f852" containerID="4b107e4a0f543fcc2d44191a99e176416c846abdaa36a32a76df3b1080fd340d" exitCode=0
Jul 09 18:32:41.345101 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.345059    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4vcw\" (UniqueName: \"kubernetes.io/projected/5f391c00-01e5-47a2-a692-81cb65a54401-kube-api-access-l4vcw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l5k64\" (UID: \"5f391c00-01e5-47a2-a692-81cb65a54401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:41.345263 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.345123    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f391c00-01e5-47a2-a692-81cb65a54401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l5k64\" (UID: \"5f391c00-01e5-47a2-a692-81cb65a54401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:41.435036 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.435012    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:41.446450 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.446392    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4vcw\" (UniqueName: \"kubernetes.io/projected/5f391c00-01e5-47a2-a692-81cb65a54401-kube-api-access-l4vcw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l5k64\" (UID: \"5f391c00-01e5-47a2-a692-81cb65a54401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:41.446450 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.446429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f391c00-01e5-47a2-a692-81cb65a54401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l5k64\" (UID: \"5f391c00-01e5-47a2-a692-81cb65a54401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:41.446754 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.446738    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f391c00-01e5-47a2-a692-81cb65a54401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l5k64\" (UID: \"5f391c00-01e5-47a2-a692-81cb65a54401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:41.455846 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.455818    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4vcw\" (UniqueName: \"kubernetes.io/projected/5f391c00-01e5-47a2-a692-81cb65a54401-kube-api-access-l4vcw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l5k64\" (UID: \"5f391c00-01e5-47a2-a692-81cb65a54401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:41.546862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.546832    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f83c532f-2092-4dc3-ae7d-0eb899f1f852-extensions-socket-volume\") pod \"f83c532f-2092-4dc3-ae7d-0eb899f1f852\" (UID: \"f83c532f-2092-4dc3-ae7d-0eb899f1f852\") "
Jul 09 18:32:41.546862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.546864    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kds6z\" (UniqueName: \"kubernetes.io/projected/f83c532f-2092-4dc3-ae7d-0eb899f1f852-kube-api-access-kds6z\") pod \"f83c532f-2092-4dc3-ae7d-0eb899f1f852\" (UID: \"f83c532f-2092-4dc3-ae7d-0eb899f1f852\") "
Jul 09 18:32:41.547316 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.547290    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83c532f-2092-4dc3-ae7d-0eb899f1f852-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f83c532f-2092-4dc3-ae7d-0eb899f1f852" (UID: "f83c532f-2092-4dc3-ae7d-0eb899f1f852"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:32:41.549060 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.549038    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83c532f-2092-4dc3-ae7d-0eb899f1f852-kube-api-access-kds6z" (OuterVolumeSpecName: "kube-api-access-kds6z") pod "f83c532f-2092-4dc3-ae7d-0eb899f1f852" (UID: "f83c532f-2092-4dc3-ae7d-0eb899f1f852"). InnerVolumeSpecName "kube-api-access-kds6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:32:41.567830 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.567789    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:41.648240 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.648216    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f83c532f-2092-4dc3-ae7d-0eb899f1f852-extensions-socket-volume\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:32:41.648338 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.648244    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kds6z\" (UniqueName: \"kubernetes.io/projected/f83c532f-2092-4dc3-ae7d-0eb899f1f852-kube-api-access-kds6z\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:32:41.704752 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:41.704725    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"]
Jul 09 18:32:41.707427 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:32:41.707402    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f391c00_01e5_47a2_a692_81cb65a54401.slice/crio-8ee47d58c30dea293b8e8d9c9843f07f730b276c528871093ccc268b3f1966dd WatchSource:0}: Error finding container 8ee47d58c30dea293b8e8d9c9843f07f730b276c528871093ccc268b3f1966dd: Status 404 returned error can't find the container with id 8ee47d58c30dea293b8e8d9c9843f07f730b276c528871093ccc268b3f1966dd
Jul 09 18:32:42.330868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:42.330834    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tt6pw"
Jul 09 18:32:42.330868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:42.330854    2566 scope.go:117] "RemoveContainer" containerID="4b107e4a0f543fcc2d44191a99e176416c846abdaa36a32a76df3b1080fd340d"
Jul 09 18:32:42.332559 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:42.332523    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64" event={"ID":"5f391c00-01e5-47a2-a692-81cb65a54401","Type":"ContainerStarted","Data":"2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a"}
Jul 09 18:32:42.332672 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:42.332583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64" event={"ID":"5f391c00-01e5-47a2-a692-81cb65a54401","Type":"ContainerStarted","Data":"8ee47d58c30dea293b8e8d9c9843f07f730b276c528871093ccc268b3f1966dd"}
Jul 09 18:32:42.332672 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:42.332612    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:42.369123 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:42.369064    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64" podStartSLOduration=1.369045525 podStartE2EDuration="1.369045525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:32:42.366767859 +0000 UTC m=+439.502585331" watchObservedRunningTime="2026-07-09 18:32:42.369045525 +0000 UTC m=+439.504862999"
Jul 09 18:32:43.364522 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:43.364486    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f83c532f-2092-4dc3-ae7d-0eb899f1f852" path="/var/lib/kubelet/pods/f83c532f-2092-4dc3-ae7d-0eb899f1f852/volumes"
Jul 09 18:32:53.339417 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:53.339340    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:58.345982 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:58.345940    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"]
Jul 09 18:32:58.346542 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:58.346254    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64" podUID="5f391c00-01e5-47a2-a692-81cb65a54401" containerName="manager" containerID="cri-o://2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a" gracePeriod=10
Jul 09 18:32:58.590873 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:58.590844    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:58.691522 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:58.691495    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f391c00-01e5-47a2-a692-81cb65a54401-extensions-socket-volume\") pod \"5f391c00-01e5-47a2-a692-81cb65a54401\" (UID: \"5f391c00-01e5-47a2-a692-81cb65a54401\") "
Jul 09 18:32:58.691640 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:58.691625    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4vcw\" (UniqueName: \"kubernetes.io/projected/5f391c00-01e5-47a2-a692-81cb65a54401-kube-api-access-l4vcw\") pod \"5f391c00-01e5-47a2-a692-81cb65a54401\" (UID: \"5f391c00-01e5-47a2-a692-81cb65a54401\") "
Jul 09 18:32:58.691864 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:58.691843    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f391c00-01e5-47a2-a692-81cb65a54401-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5f391c00-01e5-47a2-a692-81cb65a54401" (UID: "5f391c00-01e5-47a2-a692-81cb65a54401"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:32:58.693552 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:58.693528    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f391c00-01e5-47a2-a692-81cb65a54401-kube-api-access-l4vcw" (OuterVolumeSpecName: "kube-api-access-l4vcw") pod "5f391c00-01e5-47a2-a692-81cb65a54401" (UID: "5f391c00-01e5-47a2-a692-81cb65a54401"). InnerVolumeSpecName "kube-api-access-l4vcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:32:58.793036 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:58.793009    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f391c00-01e5-47a2-a692-81cb65a54401-extensions-socket-volume\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:32:58.793036 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:58.793036    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4vcw\" (UniqueName: \"kubernetes.io/projected/5f391c00-01e5-47a2-a692-81cb65a54401-kube-api-access-l4vcw\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:32:59.408575 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:59.408539    2566 generic.go:356] "Generic (PLEG): container finished" podID="5f391c00-01e5-47a2-a692-81cb65a54401" containerID="2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a" exitCode=0
Jul 09 18:32:59.408929 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:59.408595    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"
Jul 09 18:32:59.408929 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:59.408625    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64" event={"ID":"5f391c00-01e5-47a2-a692-81cb65a54401","Type":"ContainerDied","Data":"2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a"}
Jul 09 18:32:59.408929 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:59.408696    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64" event={"ID":"5f391c00-01e5-47a2-a692-81cb65a54401","Type":"ContainerDied","Data":"8ee47d58c30dea293b8e8d9c9843f07f730b276c528871093ccc268b3f1966dd"}
Jul 09 18:32:59.408929 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:59.408721    2566 scope.go:117] "RemoveContainer" containerID="2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a"
Jul 09 18:32:59.417823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:59.417805    2566 scope.go:117] "RemoveContainer" containerID="2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a"
Jul 09 18:32:59.418098 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:32:59.418077    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a\": container with ID starting with 2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a not found: ID does not exist" containerID="2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a"
Jul 09 18:32:59.418164 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:59.418106    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a"} err="failed to get container status \"2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a\": rpc error: code = NotFound desc = could not find container \"2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a\": container with ID starting with 2f60456f63a11da00a369bacbe80050c4425a3d1a9853309201dd27fcf17c46a not found: ID does not exist"
Jul 09 18:32:59.428588 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:59.428553    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"]
Jul 09 18:32:59.437963 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:32:59.437935    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l5k64"]
Jul 09 18:33:01.359856 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:01.359812    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f391c00-01e5-47a2-a692-81cb65a54401" path="/var/lib/kubelet/pods/5f391c00-01e5-47a2-a692-81cb65a54401/volumes"
Jul 09 18:33:14.677823 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.677792    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"]
Jul 09 18:33:14.678268 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.678113    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f391c00-01e5-47a2-a692-81cb65a54401" containerName="manager"
Jul 09 18:33:14.678268 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.678123    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f391c00-01e5-47a2-a692-81cb65a54401" containerName="manager"
Jul 09 18:33:14.678268 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.678183    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f391c00-01e5-47a2-a692-81cb65a54401" containerName="manager"
Jul 09 18:33:14.682001 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.681979    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.684245 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.684227    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bltfb\""
Jul 09 18:33:14.693793 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.693773    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"]
Jul 09 18:33:14.817228 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.817206    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.817342 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.817245    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.817342 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.817270    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.817432 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.817335    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.817432 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.817361    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82bhd\" (UniqueName: \"kubernetes.io/projected/03aabde7-0427-4b32-afe1-dca90a34ae24-kube-api-access-82bhd\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.817432 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.817386    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03aabde7-0427-4b32-afe1-dca90a34ae24-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.817432 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.817405    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.817560 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.817463    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.817560 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.817495    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.917832 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.917803    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.917929 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.917838    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.917929 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.917855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.917929 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.917895    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.917929 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.917918    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82bhd\" (UniqueName: \"kubernetes.io/projected/03aabde7-0427-4b32-afe1-dca90a34ae24-kube-api-access-82bhd\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.918141 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.917952    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03aabde7-0427-4b32-afe1-dca90a34ae24-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.918141 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.917976    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.918141 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.918035    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.918141 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.918071    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.918342 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.918226    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.918342 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.918303    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.918342 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.918334    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.918549 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.918524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.918750 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.918730    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03aabde7-0427-4b32-afe1-dca90a34ae24-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.920479 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.920454    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.920641 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.920621    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.926679 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.926638    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82bhd\" (UniqueName: \"kubernetes.io/projected/03aabde7-0427-4b32-afe1-dca90a34ae24-kube-api-access-82bhd\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.927943 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.927895    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03aabde7-0427-4b32-afe1-dca90a34ae24-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7dqvt\" (UID: \"03aabde7-0427-4b32-afe1-dca90a34ae24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:14.991889 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:14.991868    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:15.131119 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:15.131035    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"]
Jul 09 18:33:15.133389 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:33:15.133364    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03aabde7_0427_4b32_afe1_dca90a34ae24.slice/crio-f19c271ee0095bbc90fadc79fc9f3e7ba29143507fa8ca537a01db61a07284d7 WatchSource:0}: Error finding container f19c271ee0095bbc90fadc79fc9f3e7ba29143507fa8ca537a01db61a07284d7: Status 404 returned error can't find the container with id f19c271ee0095bbc90fadc79fc9f3e7ba29143507fa8ca537a01db61a07284d7
Jul 09 18:33:15.483711 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:15.483679    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt" event={"ID":"03aabde7-0427-4b32-afe1-dca90a34ae24","Type":"ContainerStarted","Data":"f19c271ee0095bbc90fadc79fc9f3e7ba29143507fa8ca537a01db61a07284d7"}
Jul 09 18:33:17.943290 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:17.943248    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:33:17.943563 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:17.943326    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:33:17.943563 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:17.943355    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:33:18.501881 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:18.501849    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt" event={"ID":"03aabde7-0427-4b32-afe1-dca90a34ae24","Type":"ContainerStarted","Data":"c1817261ca4749c8903f95009358f76b7cd7867047508061ead92efb2f90bc78"}
Jul 09 18:33:18.524627 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:18.524584    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt" podStartSLOduration=1.716769789 podStartE2EDuration="4.524569036s" totalImagesPullingTime="2.807799247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:33:18.523454291 +0000 UTC m=+475.659271768" watchObservedRunningTime="2026-07-09 18:33:18.524569036 +0000 UTC m=+475.660386507"
Jul 09 18:33:18.992623 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:18.992591    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:18.997471 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:18.997443    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:19.507075 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:19.507040    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:19.508108 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:19.508082    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:33:33.268904 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.268870    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"]
Jul 09 18:33:33.273590 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.273570    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:33.278205 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.278184    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-68mcm\""
Jul 09 18:33:33.278278 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.278214    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 18:33:33.278890 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.278873    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 18:33:33.280800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.280779    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 18:33:33.298951 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.298929    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"]
Jul 09 18:33:33.356407 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.356376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zksqf\" (UniqueName: \"kubernetes.io/projected/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-kube-api-access-zksqf\") pod \"limitador-limitador-64c94fd64f-7qsd9\" (UID: \"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:33.356544 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.356412    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-config-file\") pod \"limitador-limitador-64c94fd64f-7qsd9\" (UID: \"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:33.371310 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.371278    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"]
Jul 09 18:33:33.457466 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.457440    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-config-file\") pod \"limitador-limitador-64c94fd64f-7qsd9\" (UID: \"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:33.457598 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.457526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zksqf\" (UniqueName: \"kubernetes.io/projected/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-kube-api-access-zksqf\") pod \"limitador-limitador-64c94fd64f-7qsd9\" (UID: \"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:33.458106 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.458087    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-config-file\") pod \"limitador-limitador-64c94fd64f-7qsd9\" (UID: \"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:33.477130 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.477099    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zksqf\" (UniqueName: \"kubernetes.io/projected/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-kube-api-access-zksqf\") pod \"limitador-limitador-64c94fd64f-7qsd9\" (UID: \"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:33.583352 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.583288    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:33.714942 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:33.714918    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"]
Jul 09 18:33:33.717662 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:33:33.717617    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc2cc0d5_eeeb_44ad_afcb_97da8caa5b7e.slice/crio-438b66325c8854d5a1b948ff983f16780b6717affdf9497745ca0b2fe68cf5a9 WatchSource:0}: Error finding container 438b66325c8854d5a1b948ff983f16780b6717affdf9497745ca0b2fe68cf5a9: Status 404 returned error can't find the container with id 438b66325c8854d5a1b948ff983f16780b6717affdf9497745ca0b2fe68cf5a9
Jul 09 18:33:34.580076 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:34.579971    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9" event={"ID":"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e","Type":"ContainerStarted","Data":"438b66325c8854d5a1b948ff983f16780b6717affdf9497745ca0b2fe68cf5a9"}
Jul 09 18:33:36.592039 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:36.592008    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9" event={"ID":"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e","Type":"ContainerStarted","Data":"651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8"}
Jul 09 18:33:36.592336 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:36.592139    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:36.610307 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:36.610263    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9" podStartSLOduration=0.924857662 podStartE2EDuration="3.610249237s" totalImagesPullingTime="2.685391575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:33:36.609154928 +0000 UTC m=+493.744972392" watchObservedRunningTime="2026-07-09 18:33:36.610249237 +0000 UTC m=+493.746066776"
Jul 09 18:33:47.597614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:47.597582    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:51.611327 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:51.611282    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"]
Jul 09 18:33:51.611813 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:51.611490    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9" podUID="bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e" containerName="limitador" containerID="cri-o://651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8" gracePeriod=30
Jul 09 18:33:52.153610 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.153589    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:52.201422 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.201359    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-config-file\") pod \"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e\" (UID: \"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e\") "
Jul 09 18:33:52.201422 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.201394    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zksqf\" (UniqueName: \"kubernetes.io/projected/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-kube-api-access-zksqf\") pod \"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e\" (UID: \"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e\") "
Jul 09 18:33:52.201699 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.201678    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-config-file" (OuterVolumeSpecName: "config-file") pod "bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e" (UID: "bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:33:52.203297 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.203270    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-kube-api-access-zksqf" (OuterVolumeSpecName: "kube-api-access-zksqf") pod "bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e" (UID: "bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e"). InnerVolumeSpecName "kube-api-access-zksqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:33:52.302314 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.302291    2566 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-config-file\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:33:52.302314 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.302312    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zksqf\" (UniqueName: \"kubernetes.io/projected/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e-kube-api-access-zksqf\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:33:52.662218 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.662149    2566 generic.go:356] "Generic (PLEG): container finished" podID="bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e" containerID="651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8" exitCode=0
Jul 09 18:33:52.662218 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.662202    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"
Jul 09 18:33:52.662615 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.662240    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9" event={"ID":"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e","Type":"ContainerDied","Data":"651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8"}
Jul 09 18:33:52.662615 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.662300    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7qsd9" event={"ID":"bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e","Type":"ContainerDied","Data":"438b66325c8854d5a1b948ff983f16780b6717affdf9497745ca0b2fe68cf5a9"}
Jul 09 18:33:52.662615 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.662331    2566 scope.go:117] "RemoveContainer" containerID="651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8"
Jul 09 18:33:52.671772 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.671750    2566 scope.go:117] "RemoveContainer" containerID="651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8"
Jul 09 18:33:52.672039 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:33:52.672018    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8\": container with ID starting with 651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8 not found: ID does not exist" containerID="651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8"
Jul 09 18:33:52.672102 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.672054    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8"} err="failed to get container status \"651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8\": rpc error: code = NotFound desc = could not find container \"651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8\": container with ID starting with 651161eff77fbded5da63c426eab0265fe45deece56949717fb15c6ec5f378f8 not found: ID does not exist"
Jul 09 18:33:52.687669 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.685707    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"]
Jul 09 18:33:52.690718 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:52.690694    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7qsd9"]
Jul 09 18:33:53.360238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:53.360208    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e" path="/var/lib/kubelet/pods/bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e/volumes"
Jul 09 18:33:56.581317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.581289    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"]
Jul 09 18:33:56.581689 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.581640    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e" containerName="limitador"
Jul 09 18:33:56.581689 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.581668    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e" containerName="limitador"
Jul 09 18:33:56.581815 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.581771    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc2cc0d5-eeeb-44ad-afcb-97da8caa5b7e" containerName="limitador"
Jul 09 18:33:56.586559 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.586539    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"
Jul 09 18:33:56.588857 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.588836    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 18:33:56.588964 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.588906    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 09 18:33:56.588964 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.588933    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5w4zf\""
Jul 09 18:33:56.589688 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.589670    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 18:33:56.592279 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.592259    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"]
Jul 09 18:33:56.735032 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.734999    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvblx\" (UniqueName: \"kubernetes.io/projected/f83ae15b-25a1-489e-ac7d-c5c1b259d594-kube-api-access-pvblx\") pod \"postgres-69fb4594cf-s5bmd\" (UID: \"f83ae15b-25a1-489e-ac7d-c5c1b259d594\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"
Jul 09 18:33:56.735032 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.735043    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f83ae15b-25a1-489e-ac7d-c5c1b259d594-data\") pod \"postgres-69fb4594cf-s5bmd\" (UID: \"f83ae15b-25a1-489e-ac7d-c5c1b259d594\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"
Jul 09 18:33:56.836452 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.836374    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvblx\" (UniqueName: \"kubernetes.io/projected/f83ae15b-25a1-489e-ac7d-c5c1b259d594-kube-api-access-pvblx\") pod \"postgres-69fb4594cf-s5bmd\" (UID: \"f83ae15b-25a1-489e-ac7d-c5c1b259d594\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"
Jul 09 18:33:56.836452 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.836421    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f83ae15b-25a1-489e-ac7d-c5c1b259d594-data\") pod \"postgres-69fb4594cf-s5bmd\" (UID: \"f83ae15b-25a1-489e-ac7d-c5c1b259d594\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"
Jul 09 18:33:56.836827 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.836809    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f83ae15b-25a1-489e-ac7d-c5c1b259d594-data\") pod \"postgres-69fb4594cf-s5bmd\" (UID: \"f83ae15b-25a1-489e-ac7d-c5c1b259d594\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"
Jul 09 18:33:56.845687 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.845663    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvblx\" (UniqueName: \"kubernetes.io/projected/f83ae15b-25a1-489e-ac7d-c5c1b259d594-kube-api-access-pvblx\") pod \"postgres-69fb4594cf-s5bmd\" (UID: \"f83ae15b-25a1-489e-ac7d-c5c1b259d594\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"
Jul 09 18:33:56.897474 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:56.897453    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"
Jul 09 18:33:57.027364 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:57.027331    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"]
Jul 09 18:33:57.030475 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:33:57.030446    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83ae15b_25a1_489e_ac7d_c5c1b259d594.slice/crio-591f0858070365bf6e1db7b9b3a223f2e25afb8a77b5c6ba2327844ce408a737 WatchSource:0}: Error finding container 591f0858070365bf6e1db7b9b3a223f2e25afb8a77b5c6ba2327844ce408a737: Status 404 returned error can't find the container with id 591f0858070365bf6e1db7b9b3a223f2e25afb8a77b5c6ba2327844ce408a737
Jul 09 18:33:57.686583 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:33:57.686544    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd" event={"ID":"f83ae15b-25a1-489e-ac7d-c5c1b259d594","Type":"ContainerStarted","Data":"591f0858070365bf6e1db7b9b3a223f2e25afb8a77b5c6ba2327844ce408a737"}
Jul 09 18:34:02.712592 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:02.712552    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd" event={"ID":"f83ae15b-25a1-489e-ac7d-c5c1b259d594","Type":"ContainerStarted","Data":"1e18b2dc210bbc1f02bf6fd9e1250a5af3b4ad775a571443cadb5225a5657ba2"}
Jul 09 18:34:02.712971 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:02.712632    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"
Jul 09 18:34:02.732543 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:02.732502    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd" podStartSLOduration=2.029480234 podStartE2EDuration="6.732489082s" totalImagesPullingTime="4.703008848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:34:02.732397674 +0000 UTC m=+519.868215157" watchObservedRunningTime="2026-07-09 18:34:02.732489082 +0000 UTC m=+519.868306610"
Jul 09 18:34:08.746902 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:08.746873    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s5bmd"
Jul 09 18:34:30.612301 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.612269    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76df487c7b-wrxdr"]
Jul 09 18:34:30.616827 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.616811    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:30.619274 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.619253    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sxx9t\""
Jul 09 18:34:30.619383 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.619278    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 09 18:34:30.619383 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.619362    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 18:34:30.626163 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.626144    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76df487c7b-wrxdr"]
Jul 09 18:34:30.707401 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.707370    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4020b6c-a17c-4db2-a588-e0397cb86906-webhook-cert\") pod \"maas-controller-76df487c7b-wrxdr\" (UID: \"e4020b6c-a17c-4db2-a588-e0397cb86906\") " pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:30.707538 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.707409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47c59\" (UniqueName: \"kubernetes.io/projected/e4020b6c-a17c-4db2-a588-e0397cb86906-kube-api-access-47c59\") pod \"maas-controller-76df487c7b-wrxdr\" (UID: \"e4020b6c-a17c-4db2-a588-e0397cb86906\") " pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:30.775908 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.775876    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5797656c88-wwtvg"]
Jul 09 18:34:30.779508 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.779491    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:34:30.791572 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.791550    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5797656c88-wwtvg"]
Jul 09 18:34:30.808477 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.808453    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzs9t\" (UniqueName: \"kubernetes.io/projected/1b9f0881-11c5-413d-bcf6-299e7b1708a5-kube-api-access-vzs9t\") pod \"maas-controller-5797656c88-wwtvg\" (UID: \"1b9f0881-11c5-413d-bcf6-299e7b1708a5\") " pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:34:30.808569 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.808484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4020b6c-a17c-4db2-a588-e0397cb86906-webhook-cert\") pod \"maas-controller-76df487c7b-wrxdr\" (UID: \"e4020b6c-a17c-4db2-a588-e0397cb86906\") " pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:30.808569 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.808515    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47c59\" (UniqueName: \"kubernetes.io/projected/e4020b6c-a17c-4db2-a588-e0397cb86906-kube-api-access-47c59\") pod \"maas-controller-76df487c7b-wrxdr\" (UID: \"e4020b6c-a17c-4db2-a588-e0397cb86906\") " pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:30.808640 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.808600    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b9f0881-11c5-413d-bcf6-299e7b1708a5-webhook-cert\") pod \"maas-controller-5797656c88-wwtvg\" (UID: \"1b9f0881-11c5-413d-bcf6-299e7b1708a5\") " pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:34:30.808640 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:34:30.808602    2566 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 09 18:34:30.808719 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:34:30.808687    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4020b6c-a17c-4db2-a588-e0397cb86906-webhook-cert podName:e4020b6c-a17c-4db2-a588-e0397cb86906 nodeName:}" failed. No retries permitted until 2026-07-09 18:34:31.308666754 +0000 UTC m=+548.444484204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/e4020b6c-a17c-4db2-a588-e0397cb86906-webhook-cert") pod "maas-controller-76df487c7b-wrxdr" (UID: "e4020b6c-a17c-4db2-a588-e0397cb86906") : secret "maas-controller-webhook-cert" not found
Jul 09 18:34:30.817533 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.817513    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47c59\" (UniqueName: \"kubernetes.io/projected/e4020b6c-a17c-4db2-a588-e0397cb86906-kube-api-access-47c59\") pod \"maas-controller-76df487c7b-wrxdr\" (UID: \"e4020b6c-a17c-4db2-a588-e0397cb86906\") " pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:30.909829 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.909767    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b9f0881-11c5-413d-bcf6-299e7b1708a5-webhook-cert\") pod \"maas-controller-5797656c88-wwtvg\" (UID: \"1b9f0881-11c5-413d-bcf6-299e7b1708a5\") " pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:34:30.909829 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.909811    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzs9t\" (UniqueName: \"kubernetes.io/projected/1b9f0881-11c5-413d-bcf6-299e7b1708a5-kube-api-access-vzs9t\") pod \"maas-controller-5797656c88-wwtvg\" (UID: \"1b9f0881-11c5-413d-bcf6-299e7b1708a5\") " pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:34:30.912088 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.912067    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b9f0881-11c5-413d-bcf6-299e7b1708a5-webhook-cert\") pod \"maas-controller-5797656c88-wwtvg\" (UID: \"1b9f0881-11c5-413d-bcf6-299e7b1708a5\") " pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:34:30.917520 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:30.917500    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzs9t\" (UniqueName: \"kubernetes.io/projected/1b9f0881-11c5-413d-bcf6-299e7b1708a5-kube-api-access-vzs9t\") pod \"maas-controller-5797656c88-wwtvg\" (UID: \"1b9f0881-11c5-413d-bcf6-299e7b1708a5\") " pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:34:31.089416 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:31.089376    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:34:31.218851 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:31.218826    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5797656c88-wwtvg"]
Jul 09 18:34:31.220786 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:34:31.220761    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9f0881_11c5_413d_bcf6_299e7b1708a5.slice/crio-528eec966af77bb11b8877cd67ad070915914de4e79ad2defd06a957ed038c01 WatchSource:0}: Error finding container 528eec966af77bb11b8877cd67ad070915914de4e79ad2defd06a957ed038c01: Status 404 returned error can't find the container with id 528eec966af77bb11b8877cd67ad070915914de4e79ad2defd06a957ed038c01
Jul 09 18:34:31.313974 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:31.313945    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4020b6c-a17c-4db2-a588-e0397cb86906-webhook-cert\") pod \"maas-controller-76df487c7b-wrxdr\" (UID: \"e4020b6c-a17c-4db2-a588-e0397cb86906\") " pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:31.316169 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:31.316149    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4020b6c-a17c-4db2-a588-e0397cb86906-webhook-cert\") pod \"maas-controller-76df487c7b-wrxdr\" (UID: \"e4020b6c-a17c-4db2-a588-e0397cb86906\") " pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:31.527830 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:31.527750    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:31.671140 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:31.671112    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76df487c7b-wrxdr"]
Jul 09 18:34:31.673518 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:34:31.673491    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4020b6c_a17c_4db2_a588_e0397cb86906.slice/crio-5c7e2f1902566bccec0a59eb74447ab7e34ee3819f698e9d8c23f3b8ccaf6d09 WatchSource:0}: Error finding container 5c7e2f1902566bccec0a59eb74447ab7e34ee3819f698e9d8c23f3b8ccaf6d09: Status 404 returned error can't find the container with id 5c7e2f1902566bccec0a59eb74447ab7e34ee3819f698e9d8c23f3b8ccaf6d09
Jul 09 18:34:31.842272 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:31.842191    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5797656c88-wwtvg" event={"ID":"1b9f0881-11c5-413d-bcf6-299e7b1708a5","Type":"ContainerStarted","Data":"528eec966af77bb11b8877cd67ad070915914de4e79ad2defd06a957ed038c01"}
Jul 09 18:34:31.843484 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:31.843450    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76df487c7b-wrxdr" event={"ID":"e4020b6c-a17c-4db2-a588-e0397cb86906","Type":"ContainerStarted","Data":"5c7e2f1902566bccec0a59eb74447ab7e34ee3819f698e9d8c23f3b8ccaf6d09"}
Jul 09 18:34:34.859952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:34.859911    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5797656c88-wwtvg" event={"ID":"1b9f0881-11c5-413d-bcf6-299e7b1708a5","Type":"ContainerStarted","Data":"41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e"}
Jul 09 18:34:34.860389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:34.859976    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:34:34.861461 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:34.861440    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76df487c7b-wrxdr" event={"ID":"e4020b6c-a17c-4db2-a588-e0397cb86906","Type":"ContainerStarted","Data":"934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067"}
Jul 09 18:34:34.861588 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:34.861488    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:34.879264 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:34.879218    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5797656c88-wwtvg" podStartSLOduration=2.371143384 podStartE2EDuration="4.879203496s" totalImagesPullingTime="2.508060112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:34:34.877783222 +0000 UTC m=+552.013600691" watchObservedRunningTime="2026-07-09 18:34:34.879203496 +0000 UTC m=+552.015020965"
Jul 09 18:34:34.898360 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:34.898308    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76df487c7b-wrxdr" podStartSLOduration=2.835255209 podStartE2EDuration="4.898291455s" totalImagesPullingTime="2.063036246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:34:34.895720219 +0000 UTC m=+552.031537706" watchObservedRunningTime="2026-07-09 18:34:34.898291455 +0000 UTC m=+552.034108926"
Jul 09 18:34:45.871729 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:45.871700    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:34:45.872106 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:45.871763    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:45.934957 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:45.934926    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76df487c7b-wrxdr"]
Jul 09 18:34:45.935122 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:45.935096    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-76df487c7b-wrxdr" podUID="e4020b6c-a17c-4db2-a588-e0397cb86906" containerName="manager" containerID="cri-o://934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067" gracePeriod=30
Jul 09 18:34:46.178337 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.178317    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:46.244423 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.244401    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4020b6c-a17c-4db2-a588-e0397cb86906-webhook-cert\") pod \"e4020b6c-a17c-4db2-a588-e0397cb86906\" (UID: \"e4020b6c-a17c-4db2-a588-e0397cb86906\") "
Jul 09 18:34:46.244559 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.244439    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47c59\" (UniqueName: \"kubernetes.io/projected/e4020b6c-a17c-4db2-a588-e0397cb86906-kube-api-access-47c59\") pod \"e4020b6c-a17c-4db2-a588-e0397cb86906\" (UID: \"e4020b6c-a17c-4db2-a588-e0397cb86906\") "
Jul 09 18:34:46.246450 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.246414    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4020b6c-a17c-4db2-a588-e0397cb86906-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e4020b6c-a17c-4db2-a588-e0397cb86906" (UID: "e4020b6c-a17c-4db2-a588-e0397cb86906"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:34:46.246550 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.246459    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4020b6c-a17c-4db2-a588-e0397cb86906-kube-api-access-47c59" (OuterVolumeSpecName: "kube-api-access-47c59") pod "e4020b6c-a17c-4db2-a588-e0397cb86906" (UID: "e4020b6c-a17c-4db2-a588-e0397cb86906"). InnerVolumeSpecName "kube-api-access-47c59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:34:46.345465 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.345441    2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4020b6c-a17c-4db2-a588-e0397cb86906-webhook-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:34:46.345465 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.345463    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47c59\" (UniqueName: \"kubernetes.io/projected/e4020b6c-a17c-4db2-a588-e0397cb86906-kube-api-access-47c59\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:34:46.914587 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.914552    2566 generic.go:356] "Generic (PLEG): container finished" podID="e4020b6c-a17c-4db2-a588-e0397cb86906" containerID="934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067" exitCode=0
Jul 09 18:34:46.914964 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.914601    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76df487c7b-wrxdr"
Jul 09 18:34:46.914964 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.914607    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76df487c7b-wrxdr" event={"ID":"e4020b6c-a17c-4db2-a588-e0397cb86906","Type":"ContainerDied","Data":"934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067"}
Jul 09 18:34:46.914964 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.914678    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76df487c7b-wrxdr" event={"ID":"e4020b6c-a17c-4db2-a588-e0397cb86906","Type":"ContainerDied","Data":"5c7e2f1902566bccec0a59eb74447ab7e34ee3819f698e9d8c23f3b8ccaf6d09"}
Jul 09 18:34:46.914964 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.914709    2566 scope.go:117] "RemoveContainer" containerID="934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067"
Jul 09 18:34:46.923486 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.923468    2566 scope.go:117] "RemoveContainer" containerID="934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067"
Jul 09 18:34:46.923802 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:34:46.923782    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067\": container with ID starting with 934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067 not found: ID does not exist" containerID="934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067"
Jul 09 18:34:46.923882 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.923811    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067"} err="failed to get container status \"934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067\": rpc error: code = NotFound desc = could not find container \"934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067\": container with ID starting with 934b97c00eeb782b3fbea92771574348a84ac026404aa1fc929e7419aad71067 not found: ID does not exist"
Jul 09 18:34:46.936195 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.936173    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76df487c7b-wrxdr"]
Jul 09 18:34:46.939722 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:46.939701    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76df487c7b-wrxdr"]
Jul 09 18:34:47.363374 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:34:47.363278    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4020b6c-a17c-4db2-a588-e0397cb86906" path="/var/lib/kubelet/pods/e4020b6c-a17c-4db2-a588-e0397cb86906/volumes"
Jul 09 18:35:06.919180 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:06.919140    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"]
Jul 09 18:35:06.919672 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:06.919636    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4020b6c-a17c-4db2-a588-e0397cb86906" containerName="manager"
Jul 09 18:35:06.919672 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:06.919671    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4020b6c-a17c-4db2-a588-e0397cb86906" containerName="manager"
Jul 09 18:35:06.919798 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:06.919785    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4020b6c-a17c-4db2-a588-e0397cb86906" containerName="manager"
Jul 09 18:35:06.922457 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:06.922442    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:06.925435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:06.925410    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 09 18:35:06.925588 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:06.925438    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4zbc4\""
Jul 09 18:35:06.934449 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:06.934424    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"]
Jul 09 18:35:07.000090 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.000062    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40a9b75d-475b-40d7-859e-b0544b531c7d-openshift-service-ca\") pod \"maas-api-794d8cb9fb-9wqnt\" (UID: \"40a9b75d-475b-40d7-859e-b0544b531c7d\") " pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:07.000230 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.000118    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d82m\" (UniqueName: \"kubernetes.io/projected/40a9b75d-475b-40d7-859e-b0544b531c7d-kube-api-access-7d82m\") pod \"maas-api-794d8cb9fb-9wqnt\" (UID: \"40a9b75d-475b-40d7-859e-b0544b531c7d\") " pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:07.000230 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.000185    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40a9b75d-475b-40d7-859e-b0544b531c7d-maas-api-tls\") pod \"maas-api-794d8cb9fb-9wqnt\" (UID: \"40a9b75d-475b-40d7-859e-b0544b531c7d\") " pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:07.100868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.100836    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40a9b75d-475b-40d7-859e-b0544b531c7d-maas-api-tls\") pod \"maas-api-794d8cb9fb-9wqnt\" (UID: \"40a9b75d-475b-40d7-859e-b0544b531c7d\") " pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:07.101016 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.100887    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40a9b75d-475b-40d7-859e-b0544b531c7d-openshift-service-ca\") pod \"maas-api-794d8cb9fb-9wqnt\" (UID: \"40a9b75d-475b-40d7-859e-b0544b531c7d\") " pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:07.101016 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.100938    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d82m\" (UniqueName: \"kubernetes.io/projected/40a9b75d-475b-40d7-859e-b0544b531c7d-kube-api-access-7d82m\") pod \"maas-api-794d8cb9fb-9wqnt\" (UID: \"40a9b75d-475b-40d7-859e-b0544b531c7d\") " pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:07.101542 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.101524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40a9b75d-475b-40d7-859e-b0544b531c7d-openshift-service-ca\") pod \"maas-api-794d8cb9fb-9wqnt\" (UID: \"40a9b75d-475b-40d7-859e-b0544b531c7d\") " pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:07.103138 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.103120    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40a9b75d-475b-40d7-859e-b0544b531c7d-maas-api-tls\") pod \"maas-api-794d8cb9fb-9wqnt\" (UID: \"40a9b75d-475b-40d7-859e-b0544b531c7d\") " pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:07.108970 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.108943    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d82m\" (UniqueName: \"kubernetes.io/projected/40a9b75d-475b-40d7-859e-b0544b531c7d-kube-api-access-7d82m\") pod \"maas-api-794d8cb9fb-9wqnt\" (UID: \"40a9b75d-475b-40d7-859e-b0544b531c7d\") " pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:07.232266 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.232206    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:07.360329 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.360299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"]
Jul 09 18:35:07.362096 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:35:07.362070    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a9b75d_475b_40d7_859e_b0544b531c7d.slice/crio-8c5173402c6223698e8a30da56c99dada2c3c80bc464dedc0711eb6dafaa8f3a WatchSource:0}: Error finding container 8c5173402c6223698e8a30da56c99dada2c3c80bc464dedc0711eb6dafaa8f3a: Status 404 returned error can't find the container with id 8c5173402c6223698e8a30da56c99dada2c3c80bc464dedc0711eb6dafaa8f3a
Jul 09 18:35:07.552824 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.552753    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-569l6"]
Jul 09 18:35:07.556285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.556264    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-569l6"
Jul 09 18:35:07.558764 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.558743    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 09 18:35:07.558854 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.558752    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-p6tbr\""
Jul 09 18:35:07.567427 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.567407    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-569l6"]
Jul 09 18:35:07.711669 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.707019    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8032829c-1601-4456-beea-406d065a820e-plugins-config-volume\") pod \"payload-processing-58f86c7798-569l6\" (UID: \"8032829c-1601-4456-beea-406d065a820e\") " pod="openshift-ingress/payload-processing-58f86c7798-569l6"
Jul 09 18:35:07.711669 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.707087    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh9rx\" (UniqueName: \"kubernetes.io/projected/8032829c-1601-4456-beea-406d065a820e-kube-api-access-wh9rx\") pod \"payload-processing-58f86c7798-569l6\" (UID: \"8032829c-1601-4456-beea-406d065a820e\") " pod="openshift-ingress/payload-processing-58f86c7798-569l6"
Jul 09 18:35:07.807595 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.807506    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8032829c-1601-4456-beea-406d065a820e-plugins-config-volume\") pod \"payload-processing-58f86c7798-569l6\" (UID: \"8032829c-1601-4456-beea-406d065a820e\") " pod="openshift-ingress/payload-processing-58f86c7798-569l6"
Jul 09 18:35:07.807595 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.807551    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh9rx\" (UniqueName: \"kubernetes.io/projected/8032829c-1601-4456-beea-406d065a820e-kube-api-access-wh9rx\") pod \"payload-processing-58f86c7798-569l6\" (UID: \"8032829c-1601-4456-beea-406d065a820e\") " pod="openshift-ingress/payload-processing-58f86c7798-569l6"
Jul 09 18:35:07.808182 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.808158    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8032829c-1601-4456-beea-406d065a820e-plugins-config-volume\") pod \"payload-processing-58f86c7798-569l6\" (UID: \"8032829c-1601-4456-beea-406d065a820e\") " pod="openshift-ingress/payload-processing-58f86c7798-569l6"
Jul 09 18:35:07.822483 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.822458    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh9rx\" (UniqueName: \"kubernetes.io/projected/8032829c-1601-4456-beea-406d065a820e-kube-api-access-wh9rx\") pod \"payload-processing-58f86c7798-569l6\" (UID: \"8032829c-1601-4456-beea-406d065a820e\") " pod="openshift-ingress/payload-processing-58f86c7798-569l6"
Jul 09 18:35:07.867130 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.867102    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-569l6"
Jul 09 18:35:07.997272 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:07.997235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-569l6"]
Jul 09 18:35:07.999477 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:35:07.999450    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8032829c_1601_4456_beea_406d065a820e.slice/crio-55ed7e47aea4684acf53995bbd288effe5ca3394d04fbddefdb081184e38c02e WatchSource:0}: Error finding container 55ed7e47aea4684acf53995bbd288effe5ca3394d04fbddefdb081184e38c02e: Status 404 returned error can't find the container with id 55ed7e47aea4684acf53995bbd288effe5ca3394d04fbddefdb081184e38c02e
Jul 09 18:35:08.005952 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:08.005928    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-569l6" event={"ID":"8032829c-1601-4456-beea-406d065a820e","Type":"ContainerStarted","Data":"55ed7e47aea4684acf53995bbd288effe5ca3394d04fbddefdb081184e38c02e"}
Jul 09 18:35:08.006995 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:08.006976    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt" event={"ID":"40a9b75d-475b-40d7-859e-b0544b531c7d","Type":"ContainerStarted","Data":"8c5173402c6223698e8a30da56c99dada2c3c80bc464dedc0711eb6dafaa8f3a"}
Jul 09 18:35:10.018991 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:10.018952    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-569l6" event={"ID":"8032829c-1601-4456-beea-406d065a820e","Type":"ContainerStarted","Data":"89f1d75056c52d70b748ed3abff24a85dd938e8f49e574ad0a59b5c42bfb8af0"}
Jul 09 18:35:10.019418 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:10.019062    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-569l6"
Jul 09 18:35:10.020630 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:10.020607    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt" event={"ID":"40a9b75d-475b-40d7-859e-b0544b531c7d","Type":"ContainerStarted","Data":"5d9a884024c3f50e7d3975e6a52ea070fd6d466352ac31c07ad06a480f4c3ea7"}
Jul 09 18:35:10.020759 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:10.020685    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:10.037173 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:10.037131    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-569l6" podStartSLOduration=1.160573844 podStartE2EDuration="3.037118331s" totalImagesPullingTime="1.876544487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:35:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:35:10.035410608 +0000 UTC m=+587.171228081" watchObservedRunningTime="2026-07-09 18:35:10.037118331 +0000 UTC m=+587.172935801"
Jul 09 18:35:10.054121 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:10.054070    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt" podStartSLOduration=2.413228277 podStartE2EDuration="4.054053701s" totalImagesPullingTime="1.640825424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:35:10.053356059 +0000 UTC m=+587.189173534" watchObservedRunningTime="2026-07-09 18:35:10.054053701 +0000 UTC m=+587.189871173"
Jul 09 18:35:15.044897 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:15.044868    2566 generic.go:356] "Generic (PLEG): container finished" podID="03aabde7-0427-4b32-afe1-dca90a34ae24" containerID="c1817261ca4749c8903f95009358f76b7cd7867047508061ead92efb2f90bc78" exitCode=0
Jul 09 18:35:15.045238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:15.044938    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt" event={"ID":"03aabde7-0427-4b32-afe1-dca90a34ae24","Type":"ContainerDied","Data":"c1817261ca4749c8903f95009358f76b7cd7867047508061ead92efb2f90bc78"}
Jul 09 18:35:15.045295 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:15.045276    2566 scope.go:117] "RemoveContainer" containerID="c1817261ca4749c8903f95009358f76b7cd7867047508061ead92efb2f90bc78"
Jul 09 18:35:15.046253 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:15.046226    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:35:15.046328 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:15.046292    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:35:15.046328 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:15.046321    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:35:16.030710 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:16.030681    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-794d8cb9fb-9wqnt"
Jul 09 18:35:16.053891 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:16.053845    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt" event={"ID":"03aabde7-0427-4b32-afe1-dca90a34ae24","Type":"ContainerStarted","Data":"92947bb1ff5b71a70334fdeacb9f5e6ce9fac148585c0e50932ce6eeaba73fee"}
Jul 09 18:35:16.992487 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:16.992449    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:35:16.993669 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:16.993629    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7dqvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 09 18:35:16.993785 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:16.993708    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt" podUID="03aabde7-0427-4b32-afe1-dca90a34ae24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 09 18:35:17.992266 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:17.992227    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7dqvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 09 18:35:17.992687 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:17.992285    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt" podUID="03aabde7-0427-4b32-afe1-dca90a34ae24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 09 18:35:18.993069 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:18.993032    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7dqvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 09 18:35:18.993430 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:18.993086    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt" podUID="03aabde7-0427-4b32-afe1-dca90a34ae24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 09 18:35:19.507750 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:19.507721    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:35:19.996530 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:19.996502    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:35:20.078235 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:20.078209    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7dqvt"
Jul 09 18:35:21.027109 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:35:21.027079    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-569l6"
Jul 09 18:36:02.705135 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.705052    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"]
Jul 09 18:36:02.708870 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.708848    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.711536 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.711509    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 18:36:02.711669 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.711567    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 18:36:02.712561 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.712544    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m2k7l\""
Jul 09 18:36:02.712674 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.712588    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 09 18:36:02.720565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.720545    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"]
Jul 09 18:36:02.755601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.755568    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4df0c8-5427-4174-acaf-80e59edd50d0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.755751 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.755604    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.755751 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.755641    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.755751 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.755675    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.755751 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.755738    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.755928 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.755779    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qndn\" (UniqueName: \"kubernetes.io/projected/6f4df0c8-5427-4174-acaf-80e59edd50d0-kube-api-access-6qndn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.856168 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.856138    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.856287 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.856176    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.856287 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.856208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.856287 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.856235    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qndn\" (UniqueName: \"kubernetes.io/projected/6f4df0c8-5427-4174-acaf-80e59edd50d0-kube-api-access-6qndn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.856287 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.856269    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4df0c8-5427-4174-acaf-80e59edd50d0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.856287 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.856286    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.856601 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.856580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.856707 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.856583    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.856707 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.856697    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.858401 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.858379    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4df0c8-5427-4174-acaf-80e59edd50d0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.858767 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.858752    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4df0c8-5427-4174-acaf-80e59edd50d0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:02.874489 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:02.874463    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qndn\" (UniqueName: \"kubernetes.io/projected/6f4df0c8-5427-4174-acaf-80e59edd50d0-kube-api-access-6qndn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jb8bq\" (UID: \"6f4df0c8-5427-4174-acaf-80e59edd50d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:03.019037 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:03.018963    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:03.149614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:03.149588    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"]
Jul 09 18:36:03.151743 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:36:03.151716    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4df0c8_5427_4174_acaf_80e59edd50d0.slice/crio-3839ede4a1640f668539fc838692bc7d1019d50cf8cc88e74a56b11a7f7a1401 WatchSource:0}: Error finding container 3839ede4a1640f668539fc838692bc7d1019d50cf8cc88e74a56b11a7f7a1401: Status 404 returned error can't find the container with id 3839ede4a1640f668539fc838692bc7d1019d50cf8cc88e74a56b11a7f7a1401
Jul 09 18:36:03.152985 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:03.152966    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:36:03.271986 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:03.271908    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq" event={"ID":"6f4df0c8-5427-4174-acaf-80e59edd50d0","Type":"ContainerStarted","Data":"3839ede4a1640f668539fc838692bc7d1019d50cf8cc88e74a56b11a7f7a1401"}
Jul 09 18:36:04.278509 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:04.278459    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq" event={"ID":"6f4df0c8-5427-4174-acaf-80e59edd50d0","Type":"ContainerStarted","Data":"5a02bf7d803e9ed014c2c89da0f75b2e7125a500ac4daab62af3ff7f58837a97"}
Jul 09 18:36:04.300485 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:04.300433    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq" podStartSLOduration=1.261429663 podStartE2EDuration="2.300418416s" totalImagesPullingTime="1.038988753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:36:04.298460881 +0000 UTC m=+641.434278352" watchObservedRunningTime="2026-07-09 18:36:04.300418416 +0000 UTC m=+641.436235920"
Jul 09 18:36:08.474416 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.474381    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"]
Jul 09 18:36:08.478435 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.478415    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.484804 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.484573    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 09 18:36:08.494972 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.494949    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"]
Jul 09 18:36:08.599119 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.599094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.599255 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.599124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46c70108-50bc-457e-bdb6-1556e17aefe1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.599255 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.599159    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.599255 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.599199    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.599383 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.599255    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85dxg\" (UniqueName: \"kubernetes.io/projected/46c70108-50bc-457e-bdb6-1556e17aefe1-kube-api-access-85dxg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.599383 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.599281    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.700510 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.700481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.700639 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.700528    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.700639 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.700544    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46c70108-50bc-457e-bdb6-1556e17aefe1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.700639 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.700591    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.700639 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.700614    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.700898 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.700871    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85dxg\" (UniqueName: \"kubernetes.io/projected/46c70108-50bc-457e-bdb6-1556e17aefe1-kube-api-access-85dxg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.701005 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.700976    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.701005 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.701002    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.701156 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.701037    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.702797 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.702775    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46c70108-50bc-457e-bdb6-1556e17aefe1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.703029 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.703015    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46c70108-50bc-457e-bdb6-1556e17aefe1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.710810 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.710785    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85dxg\" (UniqueName: \"kubernetes.io/projected/46c70108-50bc-457e-bdb6-1556e17aefe1-kube-api-access-85dxg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx\" (UID: \"46c70108-50bc-457e-bdb6-1556e17aefe1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.788341 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.788266    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:08.916349 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:08.916322    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"]
Jul 09 18:36:08.918238 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:36:08.918213    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c70108_50bc_457e_bdb6_1556e17aefe1.slice/crio-0f632a54d7cea8e0ca23e41187a8a93e3f723e17dc35c34b5d2ec37b244e5dce WatchSource:0}: Error finding container 0f632a54d7cea8e0ca23e41187a8a93e3f723e17dc35c34b5d2ec37b244e5dce: Status 404 returned error can't find the container with id 0f632a54d7cea8e0ca23e41187a8a93e3f723e17dc35c34b5d2ec37b244e5dce
Jul 09 18:36:09.303026 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:09.302999    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx" event={"ID":"46c70108-50bc-457e-bdb6-1556e17aefe1","Type":"ContainerStarted","Data":"3fc5329e4a533f12686d596e493bb012c89fe7953b8de6a492782ad274215583"}
Jul 09 18:36:09.303166 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:09.303046    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx" event={"ID":"46c70108-50bc-457e-bdb6-1556e17aefe1","Type":"ContainerStarted","Data":"0f632a54d7cea8e0ca23e41187a8a93e3f723e17dc35c34b5d2ec37b244e5dce"}
Jul 09 18:36:09.323825 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:09.323776    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx" podStartSLOduration=1.135406377 podStartE2EDuration="1.323761886s" totalImagesPullingTime="188.355509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:36:09.321671879 +0000 UTC m=+646.457489344" watchObservedRunningTime="2026-07-09 18:36:09.323761886 +0000 UTC m=+646.459579357"
Jul 09 18:36:13.019325 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:13.019287    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:13.019762 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:13.019356    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:13.031532 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:13.031506    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:13.331656 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:13.331565    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jb8bq"
Jul 09 18:36:18.788602 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:18.788572    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:18.789009 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:18.788678    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:18.800538 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:18.800513    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:36:19.359438 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:36:19.359409    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx"
Jul 09 18:38:44.364299 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:44.364266    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5797656c88-wwtvg"]
Jul 09 18:38:44.364874 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:44.364458    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5797656c88-wwtvg" podUID="1b9f0881-11c5-413d-bcf6-299e7b1708a5" containerName="manager" containerID="cri-o://41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e" gracePeriod=30
Jul 09 18:38:44.611043 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:44.611019    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:38:44.700326 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:44.700297    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b9f0881-11c5-413d-bcf6-299e7b1708a5-webhook-cert\") pod \"1b9f0881-11c5-413d-bcf6-299e7b1708a5\" (UID: \"1b9f0881-11c5-413d-bcf6-299e7b1708a5\") "
Jul 09 18:38:44.700465 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:44.700379    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzs9t\" (UniqueName: \"kubernetes.io/projected/1b9f0881-11c5-413d-bcf6-299e7b1708a5-kube-api-access-vzs9t\") pod \"1b9f0881-11c5-413d-bcf6-299e7b1708a5\" (UID: \"1b9f0881-11c5-413d-bcf6-299e7b1708a5\") "
Jul 09 18:38:44.702146 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:44.702123    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9f0881-11c5-413d-bcf6-299e7b1708a5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1b9f0881-11c5-413d-bcf6-299e7b1708a5" (UID: "1b9f0881-11c5-413d-bcf6-299e7b1708a5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:38:44.702284 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:44.702209    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9f0881-11c5-413d-bcf6-299e7b1708a5-kube-api-access-vzs9t" (OuterVolumeSpecName: "kube-api-access-vzs9t") pod "1b9f0881-11c5-413d-bcf6-299e7b1708a5" (UID: "1b9f0881-11c5-413d-bcf6-299e7b1708a5"). InnerVolumeSpecName "kube-api-access-vzs9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:38:44.801410 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:44.801390    2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b9f0881-11c5-413d-bcf6-299e7b1708a5-webhook-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:38:44.801515 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:44.801413    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzs9t\" (UniqueName: \"kubernetes.io/projected/1b9f0881-11c5-413d-bcf6-299e7b1708a5-kube-api-access-vzs9t\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:38:45.008338 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.008275    2566 generic.go:356] "Generic (PLEG): container finished" podID="1b9f0881-11c5-413d-bcf6-299e7b1708a5" containerID="41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e" exitCode=0
Jul 09 18:38:45.008338 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.008312    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5797656c88-wwtvg"
Jul 09 18:38:45.008468 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.008347    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5797656c88-wwtvg" event={"ID":"1b9f0881-11c5-413d-bcf6-299e7b1708a5","Type":"ContainerDied","Data":"41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e"}
Jul 09 18:38:45.008468 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.008399    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5797656c88-wwtvg" event={"ID":"1b9f0881-11c5-413d-bcf6-299e7b1708a5","Type":"ContainerDied","Data":"528eec966af77bb11b8877cd67ad070915914de4e79ad2defd06a957ed038c01"}
Jul 09 18:38:45.008468 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.008422    2566 scope.go:117] "RemoveContainer" containerID="41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e"
Jul 09 18:38:45.018005 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.017988    2566 scope.go:117] "RemoveContainer" containerID="41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e"
Jul 09 18:38:45.018298 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:38:45.018278    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e\": container with ID starting with 41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e not found: ID does not exist" containerID="41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e"
Jul 09 18:38:45.018356 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.018305    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e"} err="failed to get container status \"41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e\": rpc error: code = NotFound desc = could not find container \"41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e\": container with ID starting with 41d2253ade13f74e509e37af7ba9aa6f09e2e0d7f7407fbf13e3e602f0af976e not found: ID does not exist"
Jul 09 18:38:45.030764 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.030720    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5797656c88-wwtvg"]
Jul 09 18:38:45.032846 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.032826    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5797656c88-wwtvg"]
Jul 09 18:38:45.360073 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.360005    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9f0881-11c5-413d-bcf6-299e7b1708a5" path="/var/lib/kubelet/pods/1b9f0881-11c5-413d-bcf6-299e7b1708a5/volumes"
Jul 09 18:38:45.802243 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.802214    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5797656c88-hzbk8"]
Jul 09 18:38:45.802584 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.802552    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9f0881-11c5-413d-bcf6-299e7b1708a5" containerName="manager"
Jul 09 18:38:45.802584 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.802562    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9f0881-11c5-413d-bcf6-299e7b1708a5" containerName="manager"
Jul 09 18:38:45.802686 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.802624    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9f0881-11c5-413d-bcf6-299e7b1708a5" containerName="manager"
Jul 09 18:38:45.805686 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.805670    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5797656c88-hzbk8"
Jul 09 18:38:45.808553 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.808533    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 18:38:45.809458 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.809441    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sxx9t\""
Jul 09 18:38:45.809525 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.809489    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 09 18:38:45.817987 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.817962    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5797656c88-hzbk8"]
Jul 09 18:38:45.909197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.909176    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqn2z\" (UniqueName: \"kubernetes.io/projected/e3260026-8158-4ccf-b8c0-203e39775fa1-kube-api-access-kqn2z\") pod \"maas-controller-5797656c88-hzbk8\" (UID: \"e3260026-8158-4ccf-b8c0-203e39775fa1\") " pod="opendatahub/maas-controller-5797656c88-hzbk8"
Jul 09 18:38:45.909289 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:45.909207    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3260026-8158-4ccf-b8c0-203e39775fa1-webhook-cert\") pod \"maas-controller-5797656c88-hzbk8\" (UID: \"e3260026-8158-4ccf-b8c0-203e39775fa1\") " pod="opendatahub/maas-controller-5797656c88-hzbk8"
Jul 09 18:38:46.009608 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:46.009583    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqn2z\" (UniqueName: \"kubernetes.io/projected/e3260026-8158-4ccf-b8c0-203e39775fa1-kube-api-access-kqn2z\") pod \"maas-controller-5797656c88-hzbk8\" (UID: \"e3260026-8158-4ccf-b8c0-203e39775fa1\") " pod="opendatahub/maas-controller-5797656c88-hzbk8"
Jul 09 18:38:46.009725 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:46.009634    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3260026-8158-4ccf-b8c0-203e39775fa1-webhook-cert\") pod \"maas-controller-5797656c88-hzbk8\" (UID: \"e3260026-8158-4ccf-b8c0-203e39775fa1\") " pod="opendatahub/maas-controller-5797656c88-hzbk8"
Jul 09 18:38:46.012047 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:46.012030    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3260026-8158-4ccf-b8c0-203e39775fa1-webhook-cert\") pod \"maas-controller-5797656c88-hzbk8\" (UID: \"e3260026-8158-4ccf-b8c0-203e39775fa1\") " pod="opendatahub/maas-controller-5797656c88-hzbk8"
Jul 09 18:38:46.020221 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:46.020200    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqn2z\" (UniqueName: \"kubernetes.io/projected/e3260026-8158-4ccf-b8c0-203e39775fa1-kube-api-access-kqn2z\") pod \"maas-controller-5797656c88-hzbk8\" (UID: \"e3260026-8158-4ccf-b8c0-203e39775fa1\") " pod="opendatahub/maas-controller-5797656c88-hzbk8"
Jul 09 18:38:46.115760 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:46.115708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5797656c88-hzbk8"
Jul 09 18:38:46.245347 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:46.245326    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5797656c88-hzbk8"]
Jul 09 18:38:46.247034 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:38:46.247007    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3260026_8158_4ccf_b8c0_203e39775fa1.slice/crio-13088b3a4e9d1d12d572dfecc977122a7debdb73b2a36c0ea674e00275406edf WatchSource:0}: Error finding container 13088b3a4e9d1d12d572dfecc977122a7debdb73b2a36c0ea674e00275406edf: Status 404 returned error can't find the container with id 13088b3a4e9d1d12d572dfecc977122a7debdb73b2a36c0ea674e00275406edf
Jul 09 18:38:47.022261 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:47.022227    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5797656c88-hzbk8" event={"ID":"e3260026-8158-4ccf-b8c0-203e39775fa1","Type":"ContainerStarted","Data":"c1dc13184b4dcc460e455e5516ed660e1ee1197f65b13a9eee5dd233fd958d93"}
Jul 09 18:38:47.022597 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:47.022280    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5797656c88-hzbk8" event={"ID":"e3260026-8158-4ccf-b8c0-203e39775fa1","Type":"ContainerStarted","Data":"13088b3a4e9d1d12d572dfecc977122a7debdb73b2a36c0ea674e00275406edf"}
Jul 09 18:38:47.022597 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:47.022309    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5797656c88-hzbk8"
Jul 09 18:38:47.055195 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:47.055143    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5797656c88-hzbk8" podStartSLOduration=1.4958519209999999 podStartE2EDuration="2.055127347s" totalImagesPullingTime="559.275426ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:38:47.054695287 +0000 UTC m=+804.190512756" watchObservedRunningTime="2026-07-09 18:38:47.055127347 +0000 UTC m=+804.190944817"
Jul 09 18:38:58.033369 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:38:58.033337    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5797656c88-hzbk8"
Jul 09 18:50:57.975430 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:57.975396    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"]
Jul 09 18:50:57.979303 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:57.979283    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:57.981717 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:57.981695    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-037ceac9-serving-cert\""
Jul 09 18:50:57.990128 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:57.990106    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"]
Jul 09 18:50:58.037031 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.036998    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-openshift-service-ca\") pod \"maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:58.037153 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.037059    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqtpq\" (UniqueName: \"kubernetes.io/projected/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-kube-api-access-mqtpq\") pod \"maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:58.037153 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.037084    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-maas-api-tls\") pod \"maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:58.137664 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.137618    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqtpq\" (UniqueName: \"kubernetes.io/projected/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-kube-api-access-mqtpq\") pod \"maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:58.137817 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.137681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-maas-api-tls\") pod \"maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:58.137817 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.137748    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-openshift-service-ca\") pod \"maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:58.137936 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:50:58.137858    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-serving-cert: secret "maas-api-e2e-ait-037ceac9-serving-cert" not found
Jul 09 18:50:58.137988 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:50:58.137946    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-maas-api-tls podName:2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec nodeName:}" failed. No retries permitted until 2026-07-09 18:50:58.637923781 +0000 UTC m=+1535.773741243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-maas-api-tls") pod "maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp" (UID: "2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec") : secret "maas-api-e2e-ait-037ceac9-serving-cert" not found
Jul 09 18:50:58.138404 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.138382    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-openshift-service-ca\") pod \"maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:58.153511 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.153483    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqtpq\" (UniqueName: \"kubernetes.io/projected/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-kube-api-access-mqtpq\") pod \"maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:58.641252 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.641220    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-maas-api-tls\") pod \"maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:58.643551 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.643532    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-maas-api-tls\") pod \"maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:58.890706 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:58.890674    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:50:59.257546 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:59.257514    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"]
Jul 09 18:50:59.261315 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:50:59.261291    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cd2e174_ce77_47f4_a1b4_1bfaaeac4cec.slice/crio-b8869d18d3eacd22adc7ef0513940c04eeb0a5a64e6b45cd17a0a692abd34d48 WatchSource:0}: Error finding container b8869d18d3eacd22adc7ef0513940c04eeb0a5a64e6b45cd17a0a692abd34d48: Status 404 returned error can't find the container with id b8869d18d3eacd22adc7ef0513940c04eeb0a5a64e6b45cd17a0a692abd34d48
Jul 09 18:50:59.262686 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:59.262669    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:50:59.489265 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:50:59.489237    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp" event={"ID":"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec","Type":"ContainerStarted","Data":"b8869d18d3eacd22adc7ef0513940c04eeb0a5a64e6b45cd17a0a692abd34d48"}
Jul 09 18:51:00.495839 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:00.495806    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp" event={"ID":"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec","Type":"ContainerStarted","Data":"db06dc7cada076e21cb089369d6b0bad446be63b999e1be7bb17221e40814fcb"}
Jul 09 18:51:00.496347 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:00.495866    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:51:00.546614 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:00.546567    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp" podStartSLOduration=3.110306741 podStartE2EDuration="3.546552009s" totalImagesPullingTime="436.245268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:51:00.54590851 +0000 UTC m=+1537.681725977" watchObservedRunningTime="2026-07-09 18:51:00.546552009 +0000 UTC m=+1537.682369479"
Jul 09 18:51:03.272908 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.272875    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"]
Jul 09 18:51:03.273727 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.273673    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp" podUID="2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec" containerName="maas-api" containerID="cri-o://db06dc7cada076e21cb089369d6b0bad446be63b999e1be7bb17221e40814fcb" gracePeriod=30
Jul 09 18:51:03.511375 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.511339    2566 generic.go:356] "Generic (PLEG): container finished" podID="2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec" containerID="db06dc7cada076e21cb089369d6b0bad446be63b999e1be7bb17221e40814fcb" exitCode=0
Jul 09 18:51:03.511509 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.511415    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp" event={"ID":"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec","Type":"ContainerDied","Data":"db06dc7cada076e21cb089369d6b0bad446be63b999e1be7bb17221e40814fcb"}
Jul 09 18:51:03.526136 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.526082    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:51:03.578716 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.578687    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqtpq\" (UniqueName: \"kubernetes.io/projected/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-kube-api-access-mqtpq\") pod \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") "
Jul 09 18:51:03.578837 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.578725    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-openshift-service-ca\") pod \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") "
Jul 09 18:51:03.578837 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.578799    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-maas-api-tls\") pod \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\" (UID: \"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec\") "
Jul 09 18:51:03.579082 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.579060    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec" (UID: "2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:51:03.580799 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.580779    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-kube-api-access-mqtpq" (OuterVolumeSpecName: "kube-api-access-mqtpq") pod "2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec" (UID: "2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec"). InnerVolumeSpecName "kube-api-access-mqtpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:51:03.580882 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.580780    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec" (UID: "2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:51:03.680013 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.679991    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-maas-api-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:03.680013 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.680013    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqtpq\" (UniqueName: \"kubernetes.io/projected/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-kube-api-access-mqtpq\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:03.680140 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:03.680026    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec-openshift-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:04.517425 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:04.517396    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"
Jul 09 18:51:04.517866 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:04.517424    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp" event={"ID":"2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec","Type":"ContainerDied","Data":"b8869d18d3eacd22adc7ef0513940c04eeb0a5a64e6b45cd17a0a692abd34d48"}
Jul 09 18:51:04.517866 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:04.517480    2566 scope.go:117] "RemoveContainer" containerID="db06dc7cada076e21cb089369d6b0bad446be63b999e1be7bb17221e40814fcb"
Jul 09 18:51:04.541412 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:04.541391    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"]
Jul 09 18:51:04.546352 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:04.546333    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-037ceac9-5b6b9dc574-p9cpp"]
Jul 09 18:51:05.359193 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:05.359163    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec" path="/var/lib/kubelet/pods/2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec/volumes"
Jul 09 18:51:09.888043 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.888010    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"]
Jul 09 18:51:09.888514 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.888374    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec" containerName="maas-api"
Jul 09 18:51:09.888514 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.888384    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec" containerName="maas-api"
Jul 09 18:51:09.888514 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.888464    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cd2e174-ce77-47f4-a1b4-1bfaaeac4cec" containerName="maas-api"
Jul 09 18:51:09.892882 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.892859    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:09.897216 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.897191    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d8a1de1a-gw-openshift-default-dockercfg-qbxg8\""
Jul 09 18:51:09.913732 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.913705    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"]
Jul 09 18:51:09.928891 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.928868    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-socket\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:09.928989 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.928898    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-data\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:09.928989 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.928927    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-certs\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:09.929088 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.928990    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/557364a8-0395-45d8-a1e3-720d87296c43-istio-podinfo\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:09.929088 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.929047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-istio-token\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:09.929088 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.929069    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/557364a8-0395-45d8-a1e3-720d87296c43-istiod-ca-cert\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:09.929197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.929159    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-credential-socket\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:09.929238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.929207    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6skrv\" (UniqueName: \"kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-kube-api-access-6skrv\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:09.929331 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:09.929310    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-envoy\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.029750 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.029722    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/557364a8-0395-45d8-a1e3-720d87296c43-istio-podinfo\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.029894 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.029777    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-istio-token\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.029894 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.029829    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/557364a8-0395-45d8-a1e3-720d87296c43-istiod-ca-cert\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.029894 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.029868    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-credential-socket\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.030024 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.029899    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6skrv\" (UniqueName: \"kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-kube-api-access-6skrv\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.030024 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.029960    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-envoy\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.030024 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.030011    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-socket\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.030167 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.030032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-data\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.030167 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.030078    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-certs\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.030576 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.030536    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-socket\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.030727 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.030575    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-data\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.030727 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.030622    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-credential-socket\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.030727 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.030674    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-certs\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.030918 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.030864    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/557364a8-0395-45d8-a1e3-720d87296c43-istiod-ca-cert\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.032227 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.032205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-envoy\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.032364 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.032343    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/557364a8-0395-45d8-a1e3-720d87296c43-istio-podinfo\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.040528 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.040505    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-istio-token\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.041800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.041775    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6skrv\" (UniqueName: \"kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-kube-api-access-6skrv\") pod \"e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") " pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.204145 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.204122    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:10.339019 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.338996    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"]
Jul 09 18:51:10.340988 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:51:10.340965    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod557364a8_0395_45d8_a1e3_720d87296c43.slice/crio-7cfaa2dc73d17853e3b2c6fde752bc2afb3d48907d95350366be877c3589fe8c WatchSource:0}: Error finding container 7cfaa2dc73d17853e3b2c6fde752bc2afb3d48907d95350366be877c3589fe8c: Status 404 returned error can't find the container with id 7cfaa2dc73d17853e3b2c6fde752bc2afb3d48907d95350366be877c3589fe8c
Jul 09 18:51:10.343061 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.343033    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:51:10.343114 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.343093    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:51:10.343149 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.343122    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:51:10.547412 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.547316    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr" event={"ID":"557364a8-0395-45d8-a1e3-720d87296c43","Type":"ContainerStarted","Data":"0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c"}
Jul 09 18:51:10.547412 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.547384    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr" event={"ID":"557364a8-0395-45d8-a1e3-720d87296c43","Type":"ContainerStarted","Data":"7cfaa2dc73d17853e3b2c6fde752bc2afb3d48907d95350366be877c3589fe8c"}
Jul 09 18:51:10.597224 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:10.597176    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr" podStartSLOduration=1.59716093 podStartE2EDuration="1.59716093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:51:10.594445649 +0000 UTC m=+1547.730263142" watchObservedRunningTime="2026-07-09 18:51:10.59716093 +0000 UTC m=+1547.732978401"
Jul 09 18:51:11.204800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:11.204768    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:11.209177 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:11.209157    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:11.552036 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:11.551958    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:11.552794 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:11.552773    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:15.846990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:15.846954    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"]
Jul 09 18:51:15.847454 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:15.847135    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr" podUID="557364a8-0395-45d8-a1e3-720d87296c43" containerName="istio-proxy" containerID="cri-o://0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c" gracePeriod=30
Jul 09 18:51:21.095081 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.095058    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:21.221526 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221502    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-certs\") pod \"557364a8-0395-45d8-a1e3-720d87296c43\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") "
Jul 09 18:51:21.221686 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221549    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-data\") pod \"557364a8-0395-45d8-a1e3-720d87296c43\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") "
Jul 09 18:51:21.221686 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221570    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-socket\") pod \"557364a8-0395-45d8-a1e3-720d87296c43\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") "
Jul 09 18:51:21.221686 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221594    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-credential-socket\") pod \"557364a8-0395-45d8-a1e3-720d87296c43\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") "
Jul 09 18:51:21.221686 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221627    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-envoy\") pod \"557364a8-0395-45d8-a1e3-720d87296c43\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") "
Jul 09 18:51:21.221915 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221703    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6skrv\" (UniqueName: \"kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-kube-api-access-6skrv\") pod \"557364a8-0395-45d8-a1e3-720d87296c43\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") "
Jul 09 18:51:21.221915 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221758    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/557364a8-0395-45d8-a1e3-720d87296c43-istio-podinfo\") pod \"557364a8-0395-45d8-a1e3-720d87296c43\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") "
Jul 09 18:51:21.221915 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221770    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "557364a8-0395-45d8-a1e3-720d87296c43" (UID: "557364a8-0395-45d8-a1e3-720d87296c43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:51:21.221915 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221791    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-data" (OuterVolumeSpecName: "istio-data") pod "557364a8-0395-45d8-a1e3-720d87296c43" (UID: "557364a8-0395-45d8-a1e3-720d87296c43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:51:21.221915 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221799    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/557364a8-0395-45d8-a1e3-720d87296c43-istiod-ca-cert\") pod \"557364a8-0395-45d8-a1e3-720d87296c43\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") "
Jul 09 18:51:21.221915 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221902    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-istio-token\") pod \"557364a8-0395-45d8-a1e3-720d87296c43\" (UID: \"557364a8-0395-45d8-a1e3-720d87296c43\") "
Jul 09 18:51:21.222263 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221921    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "557364a8-0395-45d8-a1e3-720d87296c43" (UID: "557364a8-0395-45d8-a1e3-720d87296c43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:51:21.222263 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.221960    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "557364a8-0395-45d8-a1e3-720d87296c43" (UID: "557364a8-0395-45d8-a1e3-720d87296c43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:51:21.222263 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.222243    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/557364a8-0395-45d8-a1e3-720d87296c43-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "557364a8-0395-45d8-a1e3-720d87296c43" (UID: "557364a8-0395-45d8-a1e3-720d87296c43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:51:21.222389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.222247    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-certs\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:21.222389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.222289    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-data\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:21.222389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.222304    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-workload-socket\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:21.222389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.222315    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-credential-socket\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:21.224155 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.224123    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "557364a8-0395-45d8-a1e3-720d87296c43" (UID: "557364a8-0395-45d8-a1e3-720d87296c43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:51:21.224271 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.224177    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-kube-api-access-6skrv" (OuterVolumeSpecName: "kube-api-access-6skrv") pod "557364a8-0395-45d8-a1e3-720d87296c43" (UID: "557364a8-0395-45d8-a1e3-720d87296c43"). InnerVolumeSpecName "kube-api-access-6skrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:51:21.224271 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.224260    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/557364a8-0395-45d8-a1e3-720d87296c43-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "557364a8-0395-45d8-a1e3-720d87296c43" (UID: "557364a8-0395-45d8-a1e3-720d87296c43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 18:51:21.224361 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.224280    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-istio-token" (OuterVolumeSpecName: "istio-token") pod "557364a8-0395-45d8-a1e3-720d87296c43" (UID: "557364a8-0395-45d8-a1e3-720d87296c43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:51:21.322913 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.322893    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/557364a8-0395-45d8-a1e3-720d87296c43-istiod-ca-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:21.322913 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.322912    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-istio-token\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:21.323044 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.322920    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/557364a8-0395-45d8-a1e3-720d87296c43-istio-envoy\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:21.323044 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.322928    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6skrv\" (UniqueName: \"kubernetes.io/projected/557364a8-0395-45d8-a1e3-720d87296c43-kube-api-access-6skrv\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:21.323044 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.322937    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/557364a8-0395-45d8-a1e3-720d87296c43-istio-podinfo\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:21.600734 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.600668    2566 generic.go:356] "Generic (PLEG): container finished" podID="557364a8-0395-45d8-a1e3-720d87296c43" containerID="0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c" exitCode=0
Jul 09 18:51:21.600734 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.600703    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"
Jul 09 18:51:21.600734 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.600727    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr" event={"ID":"557364a8-0395-45d8-a1e3-720d87296c43","Type":"ContainerDied","Data":"0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c"}
Jul 09 18:51:21.600921 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.600771    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr" event={"ID":"557364a8-0395-45d8-a1e3-720d87296c43","Type":"ContainerDied","Data":"7cfaa2dc73d17853e3b2c6fde752bc2afb3d48907d95350366be877c3589fe8c"}
Jul 09 18:51:21.600921 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.600793    2566 scope.go:117] "RemoveContainer" containerID="0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c"
Jul 09 18:51:21.609071 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.609048    2566 scope.go:117] "RemoveContainer" containerID="0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c"
Jul 09 18:51:21.609336 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:51:21.609304    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c\": container with ID starting with 0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c not found: ID does not exist" containerID="0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c"
Jul 09 18:51:21.609415 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.609347    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c"} err="failed to get container status \"0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c\": rpc error: code = NotFound desc = could not find container \"0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c\": container with ID starting with 0b169104fb3467bcb9e6d788954f59a0d5695b3cbc877d8fb5e1fe41a207c74c not found: ID does not exist"
Jul 09 18:51:21.620442 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.620418    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"]
Jul 09 18:51:21.623205 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:21.623181    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d8a1de1a-gw-openshift-default-78c9956795-98cjr"]
Jul 09 18:51:23.360303 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:23.360273    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="557364a8-0395-45d8-a1e3-720d87296c43" path="/var/lib/kubelet/pods/557364a8-0395-45d8-a1e3-720d87296c43/volumes"
Jul 09 18:51:24.195515 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.195483    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"]
Jul 09 18:51:24.195887 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.195874    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="557364a8-0395-45d8-a1e3-720d87296c43" containerName="istio-proxy"
Jul 09 18:51:24.195936 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.195889    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="557364a8-0395-45d8-a1e3-720d87296c43" containerName="istio-proxy"
Jul 09 18:51:24.195975 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.195957    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="557364a8-0395-45d8-a1e3-720d87296c43" containerName="istio-proxy"
Jul 09 18:51:24.198817 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.198795    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.201274 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.201250    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ca76790c-serving-cert\""
Jul 09 18:51:24.209678 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.209655    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"]
Jul 09 18:51:24.347878 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.347851    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-openshift-service-ca\") pod \"maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.348028 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.347890    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-maas-api-tls\") pod \"maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.348028 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.347932    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpl4m\" (UniqueName: \"kubernetes.io/projected/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-kube-api-access-fpl4m\") pod \"maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.448841 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.448779    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-openshift-service-ca\") pod \"maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.448841 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.448823    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-maas-api-tls\") pod \"maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.449222 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.448856    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpl4m\" (UniqueName: \"kubernetes.io/projected/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-kube-api-access-fpl4m\") pod \"maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.449542 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.449507    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-openshift-service-ca\") pod \"maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.451484 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.451465    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-maas-api-tls\") pod \"maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.463214 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.463188    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpl4m\" (UniqueName: \"kubernetes.io/projected/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-kube-api-access-fpl4m\") pod \"maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.509190 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.509170    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:24.651946 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:24.651924    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"]
Jul 09 18:51:24.653711 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:51:24.653682    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5acba5e_f184_4a10_82ae_b9cfa0c83c45.slice/crio-efcde043e91b0581411d8628f3bde84d65e859137b946357f6314674079984a1 WatchSource:0}: Error finding container efcde043e91b0581411d8628f3bde84d65e859137b946357f6314674079984a1: Status 404 returned error can't find the container with id efcde043e91b0581411d8628f3bde84d65e859137b946357f6314674079984a1
Jul 09 18:51:25.622200 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:25.622167    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf" event={"ID":"c5acba5e-f184-4a10-82ae-b9cfa0c83c45","Type":"ContainerStarted","Data":"0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50"}
Jul 09 18:51:25.622588 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:25.622230    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf" event={"ID":"c5acba5e-f184-4a10-82ae-b9cfa0c83c45","Type":"ContainerStarted","Data":"efcde043e91b0581411d8628f3bde84d65e859137b946357f6314674079984a1"}
Jul 09 18:51:25.622588 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:25.622264    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:25.640950 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:25.640906    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf" podStartSLOduration=1.303735853 podStartE2EDuration="1.640892365s" totalImagesPullingTime="337.156512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:51:25.639299677 +0000 UTC m=+1562.775117150" watchObservedRunningTime="2026-07-09 18:51:25.640892365 +0000 UTC m=+1562.776709836"
Jul 09 18:51:29.628577 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:29.628547    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"]
Jul 09 18:51:29.629034 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:29.628771    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf" podUID="c5acba5e-f184-4a10-82ae-b9cfa0c83c45" containerName="maas-api" containerID="cri-o://0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50" gracePeriod=30
Jul 09 18:51:29.875210 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:29.875189    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:29.993370 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:29.993334    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpl4m\" (UniqueName: \"kubernetes.io/projected/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-kube-api-access-fpl4m\") pod \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") "
Jul 09 18:51:29.993370 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:29.993378    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-maas-api-tls\") pod \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") "
Jul 09 18:51:29.993534 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:29.993459    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-openshift-service-ca\") pod \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\" (UID: \"c5acba5e-f184-4a10-82ae-b9cfa0c83c45\") "
Jul 09 18:51:29.993820 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:29.993794    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5acba5e-f184-4a10-82ae-b9cfa0c83c45" (UID: "c5acba5e-f184-4a10-82ae-b9cfa0c83c45"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:51:29.995433 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:29.995405    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5acba5e-f184-4a10-82ae-b9cfa0c83c45" (UID: "c5acba5e-f184-4a10-82ae-b9cfa0c83c45"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:51:29.995514 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:29.995449    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-kube-api-access-fpl4m" (OuterVolumeSpecName: "kube-api-access-fpl4m") pod "c5acba5e-f184-4a10-82ae-b9cfa0c83c45" (UID: "c5acba5e-f184-4a10-82ae-b9cfa0c83c45"). InnerVolumeSpecName "kube-api-access-fpl4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:51:30.094883 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.094857    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpl4m\" (UniqueName: \"kubernetes.io/projected/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-kube-api-access-fpl4m\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:30.094883 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.094878    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-maas-api-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:30.094883 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.094887    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5acba5e-f184-4a10-82ae-b9cfa0c83c45-openshift-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:30.646530 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.646501    2566 generic.go:356] "Generic (PLEG): container finished" podID="c5acba5e-f184-4a10-82ae-b9cfa0c83c45" containerID="0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50" exitCode=0
Jul 09 18:51:30.646879 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.646548    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"
Jul 09 18:51:30.646879 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.646592    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf" event={"ID":"c5acba5e-f184-4a10-82ae-b9cfa0c83c45","Type":"ContainerDied","Data":"0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50"}
Jul 09 18:51:30.646879 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.646641    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf" event={"ID":"c5acba5e-f184-4a10-82ae-b9cfa0c83c45","Type":"ContainerDied","Data":"efcde043e91b0581411d8628f3bde84d65e859137b946357f6314674079984a1"}
Jul 09 18:51:30.646879 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.646691    2566 scope.go:117] "RemoveContainer" containerID="0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50"
Jul 09 18:51:30.655902 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.655886    2566 scope.go:117] "RemoveContainer" containerID="0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50"
Jul 09 18:51:30.656112 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:51:30.656094    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50\": container with ID starting with 0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50 not found: ID does not exist" containerID="0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50"
Jul 09 18:51:30.656172 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.656121    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50"} err="failed to get container status \"0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50\": rpc error: code = NotFound desc = could not find container \"0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50\": container with ID starting with 0774a4fddda3e9fc04b9387ebbff99eef94c6c7fca419a28a08cc495e91f1f50 not found: ID does not exist"
Jul 09 18:51:30.667822 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.667799    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"]
Jul 09 18:51:30.672225 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:30.672204    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ca76790c-7ccd4d47d9-lt9sf"]
Jul 09 18:51:31.359449 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:31.359410    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5acba5e-f184-4a10-82ae-b9cfa0c83c45" path="/var/lib/kubelet/pods/c5acba5e-f184-4a10-82ae-b9cfa0c83c45/volumes"
Jul 09 18:51:44.543517 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.543487    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"]
Jul 09 18:51:44.543886 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.543858    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5acba5e-f184-4a10-82ae-b9cfa0c83c45" containerName="maas-api"
Jul 09 18:51:44.543886 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.543870    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5acba5e-f184-4a10-82ae-b9cfa0c83c45" containerName="maas-api"
Jul 09 18:51:44.543959 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.543931    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5acba5e-f184-4a10-82ae-b9cfa0c83c45" containerName="maas-api"
Jul 09 18:51:44.547901 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.547883    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:44.550909 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.550888    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-1536c57b-serving-cert\""
Jul 09 18:51:44.575912 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.575891    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"]
Jul 09 18:51:44.709096 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.709072    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-openshift-service-ca\") pod \"maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:44.709253 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.709193    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-maas-api-tls\") pod \"maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:44.709253 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.709244    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mgwt\" (UniqueName: \"kubernetes.io/projected/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-kube-api-access-7mgwt\") pod \"maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:44.809954 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.809882    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mgwt\" (UniqueName: \"kubernetes.io/projected/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-kube-api-access-7mgwt\") pod \"maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:44.809954 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.809916    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-openshift-service-ca\") pod \"maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:44.810110 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.809989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-maas-api-tls\") pod \"maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:44.810110 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:51:44.810093    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-serving-cert: secret "maas-api-e2e-derive-1536c57b-serving-cert" not found
Jul 09 18:51:44.810190 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:51:44.810153    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-maas-api-tls podName:3dcdcdb3-eb18-4302-bd89-4cb414246eb6 nodeName:}" failed. No retries permitted until 2026-07-09 18:51:45.310133991 +0000 UTC m=+1582.445951441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-maas-api-tls") pod "maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh" (UID: "3dcdcdb3-eb18-4302-bd89-4cb414246eb6") : secret "maas-api-e2e-derive-1536c57b-serving-cert" not found
Jul 09 18:51:44.810657 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.810614    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-openshift-service-ca\") pod \"maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:44.824130 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:44.824106    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mgwt\" (UniqueName: \"kubernetes.io/projected/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-kube-api-access-7mgwt\") pod \"maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:45.313936 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:45.313906    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-maas-api-tls\") pod \"maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:45.322337 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:45.322313    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-maas-api-tls\") pod \"maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:45.457362 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:45.457334    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:45.598937 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:45.598914    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"]
Jul 09 18:51:45.600929 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:51:45.600901    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dcdcdb3_eb18_4302_bd89_4cb414246eb6.slice/crio-2ec26e073029edb24485bd580dffff0080844eed9ff01fb5b19d95216437af6e WatchSource:0}: Error finding container 2ec26e073029edb24485bd580dffff0080844eed9ff01fb5b19d95216437af6e: Status 404 returned error can't find the container with id 2ec26e073029edb24485bd580dffff0080844eed9ff01fb5b19d95216437af6e
Jul 09 18:51:45.718598 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:45.718566    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh" event={"ID":"3dcdcdb3-eb18-4302-bd89-4cb414246eb6","Type":"ContainerStarted","Data":"2ec26e073029edb24485bd580dffff0080844eed9ff01fb5b19d95216437af6e"}
Jul 09 18:51:46.724810 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:46.724779    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh" event={"ID":"3dcdcdb3-eb18-4302-bd89-4cb414246eb6","Type":"ContainerStarted","Data":"846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b"}
Jul 09 18:51:46.725222 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:46.724835    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:46.744581 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:46.744539    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh" podStartSLOduration=2.441414113 podStartE2EDuration="2.744525887s" totalImagesPullingTime="303.111774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:51:46.743586497 +0000 UTC m=+1583.879403971" watchObservedRunningTime="2026-07-09 18:51:46.744525887 +0000 UTC m=+1583.880343399"
Jul 09 18:51:49.134048 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.134018    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"]
Jul 09 18:51:49.134433 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.134190    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh" podUID="3dcdcdb3-eb18-4302-bd89-4cb414246eb6" containerName="maas-api" containerID="cri-o://846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b" gracePeriod=30
Jul 09 18:51:49.378817 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.378795    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:49.547051 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.547024    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mgwt\" (UniqueName: \"kubernetes.io/projected/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-kube-api-access-7mgwt\") pod \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") "
Jul 09 18:51:49.547207 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.547077    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-openshift-service-ca\") pod \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") "
Jul 09 18:51:49.547207 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.547139    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-maas-api-tls\") pod \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\" (UID: \"3dcdcdb3-eb18-4302-bd89-4cb414246eb6\") "
Jul 09 18:51:49.547494 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.547470    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3dcdcdb3-eb18-4302-bd89-4cb414246eb6" (UID: "3dcdcdb3-eb18-4302-bd89-4cb414246eb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:51:49.549151 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.549119    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-kube-api-access-7mgwt" (OuterVolumeSpecName: "kube-api-access-7mgwt") pod "3dcdcdb3-eb18-4302-bd89-4cb414246eb6" (UID: "3dcdcdb3-eb18-4302-bd89-4cb414246eb6"). InnerVolumeSpecName "kube-api-access-7mgwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:51:49.549243 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.549163    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3dcdcdb3-eb18-4302-bd89-4cb414246eb6" (UID: "3dcdcdb3-eb18-4302-bd89-4cb414246eb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:51:49.648424 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.648399    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-maas-api-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:49.648424 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.648422    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mgwt\" (UniqueName: \"kubernetes.io/projected/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-kube-api-access-7mgwt\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:49.648555 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.648431    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dcdcdb3-eb18-4302-bd89-4cb414246eb6-openshift-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:51:49.740223 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.740196    2566 generic.go:356] "Generic (PLEG): container finished" podID="3dcdcdb3-eb18-4302-bd89-4cb414246eb6" containerID="846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b" exitCode=0
Jul 09 18:51:49.740317 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.740237    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"
Jul 09 18:51:49.740358 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.740239    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh" event={"ID":"3dcdcdb3-eb18-4302-bd89-4cb414246eb6","Type":"ContainerDied","Data":"846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b"}
Jul 09 18:51:49.740390 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.740368    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh" event={"ID":"3dcdcdb3-eb18-4302-bd89-4cb414246eb6","Type":"ContainerDied","Data":"2ec26e073029edb24485bd580dffff0080844eed9ff01fb5b19d95216437af6e"}
Jul 09 18:51:49.740422 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.740397    2566 scope.go:117] "RemoveContainer" containerID="846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b"
Jul 09 18:51:49.749207 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.749193    2566 scope.go:117] "RemoveContainer" containerID="846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b"
Jul 09 18:51:49.749463 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:51:49.749445    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b\": container with ID starting with 846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b not found: ID does not exist" containerID="846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b"
Jul 09 18:51:49.749531 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.749470    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b"} err="failed to get container status \"846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b\": rpc error: code = NotFound desc = could not find container \"846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b\": container with ID starting with 846724d11778059366265e007aac1447a0e76fe9e95e8e6eccac92223f9fa92b not found: ID does not exist"
Jul 09 18:51:49.763010 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.762986    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"]
Jul 09 18:51:49.769439 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:49.769417    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1536c57b-847d5f4cfd-589dh"]
Jul 09 18:51:51.359771 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:51:51.359738    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dcdcdb3-eb18-4302-bd89-4cb414246eb6" path="/var/lib/kubelet/pods/3dcdcdb3-eb18-4302-bd89-4cb414246eb6/volumes"
Jul 09 18:53:36.543615 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.543578    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"]
Jul 09 18:53:36.544284 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.544119    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dcdcdb3-eb18-4302-bd89-4cb414246eb6" containerName="maas-api"
Jul 09 18:53:36.544284 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.544135    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dcdcdb3-eb18-4302-bd89-4cb414246eb6" containerName="maas-api"
Jul 09 18:53:36.544284 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.544246    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dcdcdb3-eb18-4302-bd89-4cb414246eb6" containerName="maas-api"
Jul 09 18:53:36.547232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.547210    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.550046 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.550025    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7edc7c10-openshift-default-dockercfg-bhvhn\""
Jul 09 18:53:36.629297 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.629266    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"]
Jul 09 18:53:36.694001 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.693966    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-certs\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.694001 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.694002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-data\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.694238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.694053    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-credential-socket\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.694238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.694069    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istiod-ca-cert\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.694238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.694099    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-envoy\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.694238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.694117    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-podinfo\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.694238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.694139    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5lbb\" (UniqueName: \"kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-kube-api-access-n5lbb\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.694238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.694187    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-token\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.694238 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.694205    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-socket\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.794936 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.794852    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-certs\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.794936 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.794886    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-data\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.794936 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.794918    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-credential-socket\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795214 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.794941    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istiod-ca-cert\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795214 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.794974    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-envoy\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795214 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.795000    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-podinfo\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795214 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.795047    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5lbb\" (UniqueName: \"kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-kube-api-access-n5lbb\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795214 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.795082    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-token\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795214 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.795109    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-socket\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795528 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.795307    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-credential-socket\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795528 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.795382    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-data\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795528 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.795502    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-socket\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795717 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.795641    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-certs\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.795778 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.795719    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istiod-ca-cert\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.797344 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.797324    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-envoy\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.797635 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.797617    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-podinfo\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.805291 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.805268    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-token\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.806777 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.806759    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5lbb\" (UniqueName: \"kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-kube-api-access-n5lbb\") pod \"e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") " pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:36.859766 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:36.859744    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:37.008072 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:37.008047    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"]
Jul 09 18:53:37.009215 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:53:37.009180    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ed7eb2_7a60_4369_a848_3bc16c68ab48.slice/crio-851f7a956cb5d296ad3e2fec3bf0f708b1d1eac10536b5365f073c18ddefd544 WatchSource:0}: Error finding container 851f7a956cb5d296ad3e2fec3bf0f708b1d1eac10536b5365f073c18ddefd544: Status 404 returned error can't find the container with id 851f7a956cb5d296ad3e2fec3bf0f708b1d1eac10536b5365f073c18ddefd544
Jul 09 18:53:37.011580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:37.011549    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:53:37.011678 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:37.011615    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:53:37.011678 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:37.011669    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:53:37.258603 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:37.258558    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" event={"ID":"09ed7eb2-7a60-4369-a848-3bc16c68ab48","Type":"ContainerStarted","Data":"9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f"}
Jul 09 18:53:37.258868 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:37.258669    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" event={"ID":"09ed7eb2-7a60-4369-a848-3bc16c68ab48","Type":"ContainerStarted","Data":"851f7a956cb5d296ad3e2fec3bf0f708b1d1eac10536b5365f073c18ddefd544"}
Jul 09 18:53:37.282564 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:37.282509    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" podStartSLOduration=1.282494062 podStartE2EDuration="1.282494062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:53:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:53:37.280338644 +0000 UTC m=+1694.416156117" watchObservedRunningTime="2026-07-09 18:53:37.282494062 +0000 UTC m=+1694.418311533"
Jul 09 18:53:37.860320 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:37.860287    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:38.864800 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:38.864762    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:39.269215 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:39.269181    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:39.270296 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:39.270276    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:54.271334 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:54.271286    2566 patch_prober.go:28] interesting pod/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 09 18:53:54.271811 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:54.271363    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 09 18:53:54.343718 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:54.343688    2566 generic.go:356] "Generic (PLEG): container finished" podID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerID="9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f" exitCode=0
Jul 09 18:53:54.343878 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:54.343768    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" event={"ID":"09ed7eb2-7a60-4369-a848-3bc16c68ab48","Type":"ContainerDied","Data":"9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f"}
Jul 09 18:53:54.344185 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:54.344168    2566 scope.go:117] "RemoveContainer" containerID="9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f"
Jul 09 18:53:54.345449 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:54.345363    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:53:54.345540 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:54.345508    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:53:54.345580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:54.345561    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:53:55.361908 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:55.361860    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" event={"ID":"09ed7eb2-7a60-4369-a848-3bc16c68ab48","Type":"ContainerStarted","Data":"bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a"}
Jul 09 18:53:55.860567 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:55.860523    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:53:55.860802 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:55.860687    2566 patch_prober.go:28] interesting pod/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 09 18:53:55.860802 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:55.860745    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 09 18:53:56.860517 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:56.860474    2566 patch_prober.go:28] interesting pod/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 09 18:53:56.861029 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:56.860555    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 09 18:53:57.860869 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:57.860831    2566 patch_prober.go:28] interesting pod/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 09 18:53:57.861419 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:57.861386    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 09 18:53:58.860303 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:58.860262    2566 patch_prober.go:28] interesting pod/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 09 18:53:58.860502 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:58.860336    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 09 18:53:59.200436 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:59.200399    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"]
Jul 09 18:53:59.200924 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:53:59.200628    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy" containerID="cri-o://bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a" gracePeriod=30
Jul 09 18:54:04.549478 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.549456    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:54:04.735266 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735229    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-credential-socket\") pod \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") "
Jul 09 18:54:04.735436 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735280    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-podinfo\") pod \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") "
Jul 09 18:54:04.735436 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735296    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-token\") pod \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") "
Jul 09 18:54:04.735436 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735325    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5lbb\" (UniqueName: \"kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-kube-api-access-n5lbb\") pod \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") "
Jul 09 18:54:04.735436 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735347    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-envoy\") pod \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") "
Jul 09 18:54:04.735436 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735374    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-certs\") pod \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") "
Jul 09 18:54:04.735436 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735412    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istiod-ca-cert\") pod \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") "
Jul 09 18:54:04.735769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735440    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-socket\") pod \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") "
Jul 09 18:54:04.735769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735475    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-data\") pod \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\" (UID: \"09ed7eb2-7a60-4369-a848-3bc16c68ab48\") "
Jul 09 18:54:04.735769 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735556    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09ed7eb2-7a60-4369-a848-3bc16c68ab48" (UID: "09ed7eb2-7a60-4369-a848-3bc16c68ab48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:54:04.735914 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735858    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-credential-socket\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:54:04.736030 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.735914    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09ed7eb2-7a60-4369-a848-3bc16c68ab48" (UID: "09ed7eb2-7a60-4369-a848-3bc16c68ab48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:54:04.736093 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.736068    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09ed7eb2-7a60-4369-a848-3bc16c68ab48" (UID: "09ed7eb2-7a60-4369-a848-3bc16c68ab48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:54:04.736150 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.736095    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09ed7eb2-7a60-4369-a848-3bc16c68ab48" (UID: "09ed7eb2-7a60-4369-a848-3bc16c68ab48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:54:04.736318 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.736299    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-data" (OuterVolumeSpecName: "istio-data") pod "09ed7eb2-7a60-4369-a848-3bc16c68ab48" (UID: "09ed7eb2-7a60-4369-a848-3bc16c68ab48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:54:04.738052 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.738013    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09ed7eb2-7a60-4369-a848-3bc16c68ab48" (UID: "09ed7eb2-7a60-4369-a848-3bc16c68ab48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:54:04.738052 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.738034    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-token" (OuterVolumeSpecName: "istio-token") pod "09ed7eb2-7a60-4369-a848-3bc16c68ab48" (UID: "09ed7eb2-7a60-4369-a848-3bc16c68ab48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:54:04.738187 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.738054    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09ed7eb2-7a60-4369-a848-3bc16c68ab48" (UID: "09ed7eb2-7a60-4369-a848-3bc16c68ab48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 18:54:04.738187 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.738078    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-kube-api-access-n5lbb" (OuterVolumeSpecName: "kube-api-access-n5lbb") pod "09ed7eb2-7a60-4369-a848-3bc16c68ab48" (UID: "09ed7eb2-7a60-4369-a848-3bc16c68ab48"). InnerVolumeSpecName "kube-api-access-n5lbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:54:04.836204 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.836175    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-data\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:54:04.836204 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.836200    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-podinfo\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:54:04.836358 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.836213    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-token\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:54:04.836358 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.836228    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5lbb\" (UniqueName: \"kubernetes.io/projected/09ed7eb2-7a60-4369-a848-3bc16c68ab48-kube-api-access-n5lbb\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:54:04.836358 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.836240    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istio-envoy\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:54:04.836358 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.836252    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-certs\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:54:04.836358 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.836264    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09ed7eb2-7a60-4369-a848-3bc16c68ab48-istiod-ca-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:54:04.836358 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:04.836276    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09ed7eb2-7a60-4369-a848-3bc16c68ab48-workload-socket\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:54:05.406200 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.406163    2566 generic.go:356] "Generic (PLEG): container finished" podID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerID="bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a" exitCode=0
Jul 09 18:54:05.406353 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.406214    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"
Jul 09 18:54:05.406353 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.406210    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" event={"ID":"09ed7eb2-7a60-4369-a848-3bc16c68ab48","Type":"ContainerDied","Data":"bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a"}
Jul 09 18:54:05.406353 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.406337    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s" event={"ID":"09ed7eb2-7a60-4369-a848-3bc16c68ab48","Type":"ContainerDied","Data":"851f7a956cb5d296ad3e2fec3bf0f708b1d1eac10536b5365f073c18ddefd544"}
Jul 09 18:54:05.406520 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.406360    2566 scope.go:117] "RemoveContainer" containerID="bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a"
Jul 09 18:54:05.414893 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.414701    2566 scope.go:117] "RemoveContainer" containerID="9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f"
Jul 09 18:54:05.422022 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.422005    2566 scope.go:117] "RemoveContainer" containerID="bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a"
Jul 09 18:54:05.422400 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:54:05.422344    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a\": container with ID starting with bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a not found: ID does not exist" containerID="bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a"
Jul 09 18:54:05.422400 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.422387    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a"} err="failed to get container status \"bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a\": rpc error: code = NotFound desc = could not find container \"bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a\": container with ID starting with bc3b5c08e8facb8d0277fc28a27b585d28c8287fb6d49bf8abffa5fa5133e30a not found: ID does not exist"
Jul 09 18:54:05.422580 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.422408    2566 scope.go:117] "RemoveContainer" containerID="9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f"
Jul 09 18:54:05.423165 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:54:05.422777    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f\": container with ID starting with 9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f not found: ID does not exist" containerID="9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f"
Jul 09 18:54:05.423165 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.422813    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f"} err="failed to get container status \"9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f\": rpc error: code = NotFound desc = could not find container \"9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f\": container with ID starting with 9b4c9f18766871429b75dcd186953629290800767f01a3e5eaaa6b9f5ce0fd1f not found: ID does not exist"
Jul 09 18:54:05.425078 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.425060    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"]
Jul 09 18:54:05.428596 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:05.428569    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7edc7c10-openshift-default-5ff69ddf7f-xtt5s"]
Jul 09 18:54:07.359301 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:07.359267    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" path="/var/lib/kubelet/pods/09ed7eb2-7a60-4369-a848-3bc16c68ab48/volumes"
Jul 09 18:54:20.104237 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.104205    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"]
Jul 09 18:54:20.104607 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.104579    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy"
Jul 09 18:54:20.104607 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.104590    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy"
Jul 09 18:54:20.104706 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.104662    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy"
Jul 09 18:54:20.104706 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.104673    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy"
Jul 09 18:54:20.107976 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.107941    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.110808 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.110778    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8a4ed68f-openshift-default-dockercfg-gbmrj\""
Jul 09 18:54:20.120684 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.120660    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"]
Jul 09 18:54:20.153461 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.153431    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-credential-socket\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.153602 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.153498    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-data\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.153602 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.153525    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-podinfo\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.153602 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.153582    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-envoy\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.153796 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.153622    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-certs\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.153796 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.153720    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-socket\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.153796 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.153746    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-token\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.153796 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.153774    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mjf9\" (UniqueName: \"kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-kube-api-access-5mjf9\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.153965 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.153864    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istiod-ca-cert\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.254808 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.254776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istiod-ca-cert\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.254980 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.254817    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-credential-socket\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.254980 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.254840    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-data\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.254980 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.254947    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-podinfo\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.255170 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.255011    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-envoy\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.255170 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.255058    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-certs\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.255170 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.255130    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-socket\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.255170 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.255153    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-token\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.255393 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.255183    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mjf9\" (UniqueName: \"kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-kube-api-access-5mjf9\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.255393 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.255287    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-data\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.255499 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.255423    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-certs\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.255585 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.255568    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-credential-socket\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.255677 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.255599    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-socket\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.255965 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.255936    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istiod-ca-cert\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.257382 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.257358    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-envoy\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.257574 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.257556    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-podinfo\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.263379 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.263356    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mjf9\" (UniqueName: \"kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-kube-api-access-5mjf9\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.263555 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.263538    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-token\") pod \"e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") " pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.418682 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.418662    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:20.558683 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.558641    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"]
Jul 09 18:54:20.560690 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:54:20.560633    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a78488_96be_46fd_ac7e_ac0cdd7469b4.slice/crio-d5422eb5d1303aeb232ad8155b794e5e8d6c957187008417c4824aec76d18f0e WatchSource:0}: Error finding container d5422eb5d1303aeb232ad8155b794e5e8d6c957187008417c4824aec76d18f0e: Status 404 returned error can't find the container with id d5422eb5d1303aeb232ad8155b794e5e8d6c957187008417c4824aec76d18f0e
Jul 09 18:54:20.562963 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.562931    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:54:20.563037 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.562998    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:54:20.563037 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:20.563027    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:54:21.484252 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:21.484221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" event={"ID":"b3a78488-96be-46fd-ac7e-ac0cdd7469b4","Type":"ContainerStarted","Data":"8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692"}
Jul 09 18:54:21.484626 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:21.484275    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" event={"ID":"b3a78488-96be-46fd-ac7e-ac0cdd7469b4","Type":"ContainerStarted","Data":"d5422eb5d1303aeb232ad8155b794e5e8d6c957187008417c4824aec76d18f0e"}
Jul 09 18:54:21.510337 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:21.510292    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" podStartSLOduration=1.5102783450000001 podStartE2EDuration="1.510278345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:54:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:54:21.507694767 +0000 UTC m=+1738.643512240" watchObservedRunningTime="2026-07-09 18:54:21.510278345 +0000 UTC m=+1738.646095817"
Jul 09 18:54:22.419713 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:22.419682    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:22.424749 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:22.424722    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:22.488892 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:22.488867    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:22.489565 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:22.489546    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:37.490739 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:37.490673    2566 patch_prober.go:28] interesting pod/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 09 18:54:37.491104 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:37.490737    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 09 18:54:37.560736 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:37.560672    2566 generic.go:356] "Generic (PLEG): container finished" podID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerID="8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692" exitCode=0
Jul 09 18:54:37.560736 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:37.560728    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" event={"ID":"b3a78488-96be-46fd-ac7e-ac0cdd7469b4","Type":"ContainerDied","Data":"8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692"}
Jul 09 18:54:37.561132 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:37.561114    2566 scope.go:117] "RemoveContainer" containerID="8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692"
Jul 09 18:54:37.562173 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:37.562144    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:54:37.562232 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:37.562209    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:54:37.562268 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:37.562244    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:54:38.569159 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:38.569128    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" event={"ID":"b3a78488-96be-46fd-ac7e-ac0cdd7469b4","Type":"ContainerStarted","Data":"07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2"}
Jul 09 18:54:39.419865 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:39.419829    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:39.420073 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:39.419993    2566 patch_prober.go:28] interesting pod/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 09 18:54:39.420073 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:39.420049    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 09 18:54:40.419795 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:40.419755    2566 patch_prober.go:28] interesting pod/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 09 18:54:40.420263 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:40.419811    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 09 18:54:41.419154 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:41.419123    2566 patch_prober.go:28] interesting pod/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 09 18:54:41.419323 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:41.419183    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 09 18:54:42.422666 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:42.422616    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:42.423096 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:42.423082    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:54:42.423679 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:54:42.423661    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:56:16.699688 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:16.699640    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"]
Jul 09 18:56:16.700177 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:16.699846    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy" containerID="cri-o://07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2" gracePeriod=30
Jul 09 18:56:21.954201 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:21.954179    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:56:22.057141 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057074    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-certs\") pod \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") "
Jul 09 18:56:22.057141 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057118    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-credential-socket\") pod \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") "
Jul 09 18:56:22.057141 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057136    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mjf9\" (UniqueName: \"kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-kube-api-access-5mjf9\") pod \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") "
Jul 09 18:56:22.057389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057172    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-data\") pod \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") "
Jul 09 18:56:22.057389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057191    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-podinfo\") pod \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") "
Jul 09 18:56:22.057389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057204    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-token\") pod \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") "
Jul 09 18:56:22.057389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057321    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b3a78488-96be-46fd-ac7e-ac0cdd7469b4" (UID: "b3a78488-96be-46fd-ac7e-ac0cdd7469b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:56:22.057389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057339    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b3a78488-96be-46fd-ac7e-ac0cdd7469b4" (UID: "b3a78488-96be-46fd-ac7e-ac0cdd7469b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:56:22.057389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057385    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-socket\") pod \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") "
Jul 09 18:56:22.057736 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057449    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-envoy\") pod \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") "
Jul 09 18:56:22.057736 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057480    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istiod-ca-cert\") pod \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\" (UID: \"b3a78488-96be-46fd-ac7e-ac0cdd7469b4\") "
Jul 09 18:56:22.057736 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057541    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b3a78488-96be-46fd-ac7e-ac0cdd7469b4" (UID: "b3a78488-96be-46fd-ac7e-ac0cdd7469b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:56:22.057901 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057834    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-socket\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:56:22.057901 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057856    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-workload-certs\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:56:22.057901 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057870    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-credential-socket\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:56:22.058061 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.057921    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b3a78488-96be-46fd-ac7e-ac0cdd7469b4" (UID: "b3a78488-96be-46fd-ac7e-ac0cdd7469b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:56:22.058061 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.058022    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-data" (OuterVolumeSpecName: "istio-data") pod "b3a78488-96be-46fd-ac7e-ac0cdd7469b4" (UID: "b3a78488-96be-46fd-ac7e-ac0cdd7469b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:56:22.059418 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.059386    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-token" (OuterVolumeSpecName: "istio-token") pod "b3a78488-96be-46fd-ac7e-ac0cdd7469b4" (UID: "b3a78488-96be-46fd-ac7e-ac0cdd7469b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:56:22.059522 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.059411    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-kube-api-access-5mjf9" (OuterVolumeSpecName: "kube-api-access-5mjf9") pod "b3a78488-96be-46fd-ac7e-ac0cdd7469b4" (UID: "b3a78488-96be-46fd-ac7e-ac0cdd7469b4"). InnerVolumeSpecName "kube-api-access-5mjf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:56:22.059522 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.059450    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b3a78488-96be-46fd-ac7e-ac0cdd7469b4" (UID: "b3a78488-96be-46fd-ac7e-ac0cdd7469b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 18:56:22.059595 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.059522    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b3a78488-96be-46fd-ac7e-ac0cdd7469b4" (UID: "b3a78488-96be-46fd-ac7e-ac0cdd7469b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:56:22.074789 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.074761    2566 generic.go:356] "Generic (PLEG): container finished" podID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerID="07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2" exitCode=0
Jul 09 18:56:22.074892 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.074803    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"
Jul 09 18:56:22.074892 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.074834    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" event={"ID":"b3a78488-96be-46fd-ac7e-ac0cdd7469b4","Type":"ContainerDied","Data":"07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2"}
Jul 09 18:56:22.074892 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.074884    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25" event={"ID":"b3a78488-96be-46fd-ac7e-ac0cdd7469b4","Type":"ContainerDied","Data":"d5422eb5d1303aeb232ad8155b794e5e8d6c957187008417c4824aec76d18f0e"}
Jul 09 18:56:22.075004 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.074912    2566 scope.go:117] "RemoveContainer" containerID="07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2"
Jul 09 18:56:22.083985 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.083969    2566 scope.go:117] "RemoveContainer" containerID="8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692"
Jul 09 18:56:22.091795 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.091781    2566 scope.go:117] "RemoveContainer" containerID="07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2"
Jul 09 18:56:22.092030 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:56:22.092013    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2\": container with ID starting with 07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2 not found: ID does not exist" containerID="07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2"
Jul 09 18:56:22.092075 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.092036    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2"} err="failed to get container status \"07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2\": rpc error: code = NotFound desc = could not find container \"07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2\": container with ID starting with 07a64bfbd1b49b0d7f186b2bdfc2f887905a23dd10a4ba987f2b2b7164c272e2 not found: ID does not exist"
Jul 09 18:56:22.092075 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.092050    2566 scope.go:117] "RemoveContainer" containerID="8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692"
Jul 09 18:56:22.092239 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:56:22.092220    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692\": container with ID starting with 8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692 not found: ID does not exist" containerID="8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692"
Jul 09 18:56:22.092284 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.092246    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692"} err="failed to get container status \"8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692\": rpc error: code = NotFound desc = could not find container \"8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692\": container with ID starting with 8ea773370d3f26f37a2328e1d7206ce7b597ec948f46e78cea1fa76f25ae9692 not found: ID does not exist"
Jul 09 18:56:22.124418 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.124397    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"]
Jul 09 18:56:22.136771 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.136752    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8a4ed68f-openshift-default-79b56454c5-fqh25"]
Jul 09 18:56:22.158770 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.158747    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-envoy\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:56:22.158770 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.158767    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istiod-ca-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:56:22.158888 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.158775    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mjf9\" (UniqueName: \"kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-kube-api-access-5mjf9\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:56:22.158888 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.158783    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-data\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:56:22.158888 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.158792    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-podinfo\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:56:22.158888 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:22.158801    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a78488-96be-46fd-ac7e-ac0cdd7469b4-istio-token\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:56:23.361113 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:56:23.361079    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" path="/var/lib/kubelet/pods/b3a78488-96be-46fd-ac7e-ac0cdd7469b4/volumes"
Jul 09 18:57:17.822040 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.822005    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"]
Jul 09 18:57:17.822828 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.822538    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy"
Jul 09 18:57:17.822828 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.822558    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ed7eb2-7a60-4369-a848-3bc16c68ab48" containerName="istio-proxy"
Jul 09 18:57:17.822828 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.822573    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy"
Jul 09 18:57:17.822828 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.822583    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy"
Jul 09 18:57:17.822828 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.822628    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy"
Jul 09 18:57:17.822828 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.822636    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy"
Jul 09 18:57:17.822828 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.822744    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy"
Jul 09 18:57:17.826030 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.826008    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:17.830035 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.830017    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-15a9af-serving-cert\""
Jul 09 18:57:17.842011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.841990    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"]
Jul 09 18:57:17.901667 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.899157    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zs5p\" (UniqueName: \"kubernetes.io/projected/94548f03-040d-4884-847b-211cc29958d8-kube-api-access-8zs5p\") pod \"maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:17.901667 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.899277    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94548f03-040d-4884-847b-211cc29958d8-maas-api-tls\") pod \"maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:17.901667 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.899338    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94548f03-040d-4884-847b-211cc29958d8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:17.999875 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.999848    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zs5p\" (UniqueName: \"kubernetes.io/projected/94548f03-040d-4884-847b-211cc29958d8-kube-api-access-8zs5p\") pod \"maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:17.999999 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.999908    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94548f03-040d-4884-847b-211cc29958d8-maas-api-tls\") pod \"maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:17.999999 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:17.999934    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94548f03-040d-4884-847b-211cc29958d8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:18.000089 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:57:18.000062    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-serving-cert: secret "maas-api-e2e-inf-a-15a9af-serving-cert" not found
Jul 09 18:57:18.000143 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:57:18.000133    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94548f03-040d-4884-847b-211cc29958d8-maas-api-tls podName:94548f03-040d-4884-847b-211cc29958d8 nodeName:}" failed. No retries permitted until 2026-07-09 18:57:18.500116421 +0000 UTC m=+1915.635933884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/94548f03-040d-4884-847b-211cc29958d8-maas-api-tls") pod "maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj" (UID: "94548f03-040d-4884-847b-211cc29958d8") : secret "maas-api-e2e-inf-a-15a9af-serving-cert" not found
Jul 09 18:57:18.000471 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:18.000455    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94548f03-040d-4884-847b-211cc29958d8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:18.013551 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:18.013518    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zs5p\" (UniqueName: \"kubernetes.io/projected/94548f03-040d-4884-847b-211cc29958d8-kube-api-access-8zs5p\") pod \"maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:18.503703 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:18.503672    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94548f03-040d-4884-847b-211cc29958d8-maas-api-tls\") pod \"maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:18.506071 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:18.506048    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94548f03-040d-4884-847b-211cc29958d8-maas-api-tls\") pod \"maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:18.736361 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:18.736336    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:18.881179 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:18.881144    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"]
Jul 09 18:57:18.882435 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:57:18.882402    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94548f03_040d_4884_847b_211cc29958d8.slice/crio-cd0844cd60bd2a4ab02d7ad4b0d5965b3daca4996e9d35ccef5a6e925068eb23 WatchSource:0}: Error finding container cd0844cd60bd2a4ab02d7ad4b0d5965b3daca4996e9d35ccef5a6e925068eb23: Status 404 returned error can't find the container with id cd0844cd60bd2a4ab02d7ad4b0d5965b3daca4996e9d35ccef5a6e925068eb23
Jul 09 18:57:18.883774 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:18.883754    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:57:19.352161 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:19.352127    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj" event={"ID":"94548f03-040d-4884-847b-211cc29958d8","Type":"ContainerStarted","Data":"a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106"}
Jul 09 18:57:19.352309 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:19.352192    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj" event={"ID":"94548f03-040d-4884-847b-211cc29958d8","Type":"ContainerStarted","Data":"cd0844cd60bd2a4ab02d7ad4b0d5965b3daca4996e9d35ccef5a6e925068eb23"}
Jul 09 18:57:19.352309 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:19.352222    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:19.380686 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:19.380628    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj" podStartSLOduration=1.991545691 podStartE2EDuration="2.380613098s" totalImagesPullingTime="389.067407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:57:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:57:19.378695543 +0000 UTC m=+1916.514513017" watchObservedRunningTime="2026-07-09 18:57:19.380613098 +0000 UTC m=+1916.516430570"
Jul 09 18:57:25.363709 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.363679    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:57:25.800492 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.800462    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"]
Jul 09 18:57:25.800953 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.800938    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3a78488-96be-46fd-ac7e-ac0cdd7469b4" containerName="istio-proxy"
Jul 09 18:57:25.804163 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.804143    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:25.806586 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.806567    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-86d225-serving-cert\""
Jul 09 18:57:25.819021 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.819000    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"]
Jul 09 18:57:25.863258 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.863235    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-openshift-service-ca\") pod \"maas-api-e2e-inf-b-86d225-795964994d-wv4xg\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:25.863368 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.863263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfnh6\" (UniqueName: \"kubernetes.io/projected/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-kube-api-access-bfnh6\") pod \"maas-api-e2e-inf-b-86d225-795964994d-wv4xg\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:25.863413 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.863369    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-maas-api-tls\") pod \"maas-api-e2e-inf-b-86d225-795964994d-wv4xg\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:25.964593 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.964558    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-maas-api-tls\") pod \"maas-api-e2e-inf-b-86d225-795964994d-wv4xg\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:25.964743 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.964672    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-openshift-service-ca\") pod \"maas-api-e2e-inf-b-86d225-795964994d-wv4xg\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:25.964743 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.964702    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnh6\" (UniqueName: \"kubernetes.io/projected/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-kube-api-access-bfnh6\") pod \"maas-api-e2e-inf-b-86d225-795964994d-wv4xg\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:25.965285 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.965266    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-openshift-service-ca\") pod \"maas-api-e2e-inf-b-86d225-795964994d-wv4xg\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:25.966964 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.966944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-maas-api-tls\") pod \"maas-api-e2e-inf-b-86d225-795964994d-wv4xg\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:25.973997 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:25.973978    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfnh6\" (UniqueName: \"kubernetes.io/projected/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-kube-api-access-bfnh6\") pod \"maas-api-e2e-inf-b-86d225-795964994d-wv4xg\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:26.117712 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:26.117621    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:26.270596 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:26.270572    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"]
Jul 09 18:57:26.271187 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:57:26.271159    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d3e1cc_db17_40d5_9c9c_9766a3526a28.slice/crio-fb9c442ae8b5005f1568f1c1e78ca8271aa2c0ea7eacfdf1e8fd798e3d43e701 WatchSource:0}: Error finding container fb9c442ae8b5005f1568f1c1e78ca8271aa2c0ea7eacfdf1e8fd798e3d43e701: Status 404 returned error can't find the container with id fb9c442ae8b5005f1568f1c1e78ca8271aa2c0ea7eacfdf1e8fd798e3d43e701
Jul 09 18:57:26.388546 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:26.388485    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg" event={"ID":"a4d3e1cc-db17-40d5-9c9c-9766a3526a28","Type":"ContainerStarted","Data":"fb9c442ae8b5005f1568f1c1e78ca8271aa2c0ea7eacfdf1e8fd798e3d43e701"}
Jul 09 18:57:27.394605 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:27.394570    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg" event={"ID":"a4d3e1cc-db17-40d5-9c9c-9766a3526a28","Type":"ContainerStarted","Data":"51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03"}
Jul 09 18:57:27.395038 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:27.394772    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:57:27.417637 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:27.417586    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg" podStartSLOduration=2.070090621 podStartE2EDuration="2.417574572s" totalImagesPullingTime="347.483951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:57:27.415997711 +0000 UTC m=+1924.551815187" watchObservedRunningTime="2026-07-09 18:57:27.417574572 +0000 UTC m=+1924.553392043"
Jul 09 18:57:33.404511 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:57:33.404481    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:58:41.147280 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.147249    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"]
Jul 09 18:58:41.147855 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.147457    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj" podUID="94548f03-040d-4884-847b-211cc29958d8" containerName="maas-api" containerID="cri-o://a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106" gracePeriod=30
Jul 09 18:58:41.394542 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.394513    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:58:41.484331 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.484308    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zs5p\" (UniqueName: \"kubernetes.io/projected/94548f03-040d-4884-847b-211cc29958d8-kube-api-access-8zs5p\") pod \"94548f03-040d-4884-847b-211cc29958d8\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") "
Jul 09 18:58:41.484467 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.484376    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94548f03-040d-4884-847b-211cc29958d8-openshift-service-ca\") pod \"94548f03-040d-4884-847b-211cc29958d8\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") "
Jul 09 18:58:41.484467 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.484408    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94548f03-040d-4884-847b-211cc29958d8-maas-api-tls\") pod \"94548f03-040d-4884-847b-211cc29958d8\" (UID: \"94548f03-040d-4884-847b-211cc29958d8\") "
Jul 09 18:58:41.484802 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.484778    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94548f03-040d-4884-847b-211cc29958d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "94548f03-040d-4884-847b-211cc29958d8" (UID: "94548f03-040d-4884-847b-211cc29958d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:58:41.486138 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.486109    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94548f03-040d-4884-847b-211cc29958d8-kube-api-access-8zs5p" (OuterVolumeSpecName: "kube-api-access-8zs5p") pod "94548f03-040d-4884-847b-211cc29958d8" (UID: "94548f03-040d-4884-847b-211cc29958d8"). InnerVolumeSpecName "kube-api-access-8zs5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:58:41.486301 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.486284    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94548f03-040d-4884-847b-211cc29958d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "94548f03-040d-4884-847b-211cc29958d8" (UID: "94548f03-040d-4884-847b-211cc29958d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:58:41.585983 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.585953    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zs5p\" (UniqueName: \"kubernetes.io/projected/94548f03-040d-4884-847b-211cc29958d8-kube-api-access-8zs5p\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:58:41.585983 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.585980    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94548f03-040d-4884-847b-211cc29958d8-openshift-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:58:41.586171 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.585996    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94548f03-040d-4884-847b-211cc29958d8-maas-api-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:58:41.741632 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.741556    2566 generic.go:356] "Generic (PLEG): container finished" podID="94548f03-040d-4884-847b-211cc29958d8" containerID="a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106" exitCode=0
Jul 09 18:58:41.741632 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.741602    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"
Jul 09 18:58:41.741632 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.741617    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj" event={"ID":"94548f03-040d-4884-847b-211cc29958d8","Type":"ContainerDied","Data":"a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106"}
Jul 09 18:58:41.741872 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.741679    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj" event={"ID":"94548f03-040d-4884-847b-211cc29958d8","Type":"ContainerDied","Data":"cd0844cd60bd2a4ab02d7ad4b0d5965b3daca4996e9d35ccef5a6e925068eb23"}
Jul 09 18:58:41.741872 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.741700    2566 scope.go:117] "RemoveContainer" containerID="a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106"
Jul 09 18:58:41.750559 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.750541    2566 scope.go:117] "RemoveContainer" containerID="a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106"
Jul 09 18:58:41.750829 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:58:41.750810    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106\": container with ID starting with a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106 not found: ID does not exist" containerID="a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106"
Jul 09 18:58:41.750885 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.750838    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106"} err="failed to get container status \"a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106\": rpc error: code = NotFound desc = could not find container \"a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106\": container with ID starting with a5320bed8b9949311d42c755411462e466d9ff90f749b328059a26c27e326106 not found: ID does not exist"
Jul 09 18:58:41.774215 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.774185    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"]
Jul 09 18:58:41.782223 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:41.782201    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15a9af-7cc8fd95f-fk2jj"]
Jul 09 18:58:43.360410 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:58:43.360375    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94548f03-040d-4884-847b-211cc29958d8" path="/var/lib/kubelet/pods/94548f03-040d-4884-847b-211cc29958d8/volumes"
Jul 09 18:59:25.097560 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.097529    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"]
Jul 09 18:59:25.098015 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.097759    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg" podUID="a4d3e1cc-db17-40d5-9c9c-9766a3526a28" containerName="maas-api" containerID="cri-o://51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03" gracePeriod=30
Jul 09 18:59:25.343562 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.343538    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:59:25.534288 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.534257    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-openshift-service-ca\") pod \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") "
Jul 09 18:59:25.534476 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.534336    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfnh6\" (UniqueName: \"kubernetes.io/projected/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-kube-api-access-bfnh6\") pod \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") "
Jul 09 18:59:25.534476 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.534367    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-maas-api-tls\") pod \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\" (UID: \"a4d3e1cc-db17-40d5-9c9c-9766a3526a28\") "
Jul 09 18:59:25.534720 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.534635    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a4d3e1cc-db17-40d5-9c9c-9766a3526a28" (UID: "a4d3e1cc-db17-40d5-9c9c-9766a3526a28"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:59:25.534863 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.534808    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-openshift-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:59:25.536372 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.536351    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-kube-api-access-bfnh6" (OuterVolumeSpecName: "kube-api-access-bfnh6") pod "a4d3e1cc-db17-40d5-9c9c-9766a3526a28" (UID: "a4d3e1cc-db17-40d5-9c9c-9766a3526a28"). InnerVolumeSpecName "kube-api-access-bfnh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:59:25.536514 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.536500    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a4d3e1cc-db17-40d5-9c9c-9766a3526a28" (UID: "a4d3e1cc-db17-40d5-9c9c-9766a3526a28"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:59:25.635700 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.635675    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfnh6\" (UniqueName: \"kubernetes.io/projected/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-kube-api-access-bfnh6\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:59:25.635700 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.635695    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4d3e1cc-db17-40d5-9c9c-9766a3526a28-maas-api-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:59:25.944998 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.944963    2566 generic.go:356] "Generic (PLEG): container finished" podID="a4d3e1cc-db17-40d5-9c9c-9766a3526a28" containerID="51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03" exitCode=0
Jul 09 18:59:25.945165 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.945007    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"
Jul 09 18:59:25.945165 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.945046    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg" event={"ID":"a4d3e1cc-db17-40d5-9c9c-9766a3526a28","Type":"ContainerDied","Data":"51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03"}
Jul 09 18:59:25.945165 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.945098    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg" event={"ID":"a4d3e1cc-db17-40d5-9c9c-9766a3526a28","Type":"ContainerDied","Data":"fb9c442ae8b5005f1568f1c1e78ca8271aa2c0ea7eacfdf1e8fd798e3d43e701"}
Jul 09 18:59:25.945165 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.945120    2566 scope.go:117] "RemoveContainer" containerID="51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03"
Jul 09 18:59:25.954105 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.954090    2566 scope.go:117] "RemoveContainer" containerID="51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03"
Jul 09 18:59:25.954332 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:59:25.954312    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03\": container with ID starting with 51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03 not found: ID does not exist" containerID="51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03"
Jul 09 18:59:25.954375 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.954342    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03"} err="failed to get container status \"51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03\": rpc error: code = NotFound desc = could not find container \"51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03\": container with ID starting with 51dc77a3e22020029093d91ddaaf8b4051bd082d0d21f0977f0e4fa968a79c03 not found: ID does not exist"
Jul 09 18:59:25.989510 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:25.989486    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"]
Jul 09 18:59:26.000951 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:26.000931    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-86d225-795964994d-wv4xg"]
Jul 09 18:59:27.359417 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:27.359382    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4d3e1cc-db17-40d5-9c9c-9766a3526a28" path="/var/lib/kubelet/pods/a4d3e1cc-db17-40d5-9c9c-9766a3526a28/volumes"
Jul 09 18:59:32.794307 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.794276    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"]
Jul 09 18:59:32.794760 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.794655    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94548f03-040d-4884-847b-211cc29958d8" containerName="maas-api"
Jul 09 18:59:32.794760 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.794667    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94548f03-040d-4884-847b-211cc29958d8" containerName="maas-api"
Jul 09 18:59:32.794760 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.794684    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4d3e1cc-db17-40d5-9c9c-9766a3526a28" containerName="maas-api"
Jul 09 18:59:32.794760 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.794689    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d3e1cc-db17-40d5-9c9c-9766a3526a28" containerName="maas-api"
Jul 09 18:59:32.794760 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.794757    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="94548f03-040d-4884-847b-211cc29958d8" containerName="maas-api"
Jul 09 18:59:32.794943 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.794769    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4d3e1cc-db17-40d5-9c9c-9766a3526a28" containerName="maas-api"
Jul 09 18:59:32.798044 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.798027    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.800811 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.800791    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f5e743-openshift-default-dockercfg-dcdcb\""
Jul 09 18:59:32.823836 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.823817    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"]
Jul 09 18:59:32.891714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.891681    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-podinfo\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.891862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.891729    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-certs\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.891862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.891798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istiod-ca-cert\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.891862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.891837    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-socket\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.891862 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.891858    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-credential-socket\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.892074 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.891879    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-envoy\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.892074 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.891979    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mwx6\" (UniqueName: \"kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-kube-api-access-8mwx6\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.892074 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.892020    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-data\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.892074 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.892051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-token\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.992712 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.992683    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-envoy\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.992867 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.992731    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mwx6\" (UniqueName: \"kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-kube-api-access-8mwx6\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.992867 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.992841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-data\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.992985 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.992871    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-token\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.992985 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.992896    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-podinfo\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.992985 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.992932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-certs\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.992985 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.992978    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istiod-ca-cert\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.993193 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.993031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-socket\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.993193 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.993079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-credential-socket\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.993308 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.993227    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-data\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.993308 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.993297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-certs\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.993422 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.993366    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-socket\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.993480 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.993445    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-credential-socket\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.993747 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.993725    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istiod-ca-cert\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.995070 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.995049    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-envoy\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:32.995340 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:32.995321    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-podinfo\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:33.010543 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:33.010519    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-token\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:33.010621 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:33.010529    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mwx6\" (UniqueName: \"kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-kube-api-access-8mwx6\") pod \"e2e-api-a-f5e743-openshift-default-6659477bc-5qp27\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") " pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:33.108928 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:33.108869    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:33.253177 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:33.253146    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"]
Jul 09 18:59:33.254842 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:59:33.254809    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a5a6723_1fec_4545_9f1a_7852bcdae2df.slice/crio-3828a9bfb782eb196b2e3d353574c15c3395e3bbb81b4fa502ee752d7cb34985 WatchSource:0}: Error finding container 3828a9bfb782eb196b2e3d353574c15c3395e3bbb81b4fa502ee752d7cb34985: Status 404 returned error can't find the container with id 3828a9bfb782eb196b2e3d353574c15c3395e3bbb81b4fa502ee752d7cb34985
Jul 09 18:59:33.256936 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:33.256903    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:59:33.257011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:33.256971    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:59:33.257011 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:33.257000    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:59:33.984870 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:33.984831    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" event={"ID":"2a5a6723-1fec-4545-9f1a-7852bcdae2df","Type":"ContainerStarted","Data":"d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061"}
Jul 09 18:59:33.985344 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:33.984895    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" event={"ID":"2a5a6723-1fec-4545-9f1a-7852bcdae2df","Type":"ContainerStarted","Data":"3828a9bfb782eb196b2e3d353574c15c3395e3bbb81b4fa502ee752d7cb34985"}
Jul 09 18:59:34.013621 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:34.013567    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" podStartSLOduration=2.013552811 podStartE2EDuration="2.013552811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:34.011932047 +0000 UTC m=+2051.147749519" watchObservedRunningTime="2026-07-09 18:59:34.013552811 +0000 UTC m=+2051.149370280"
Jul 09 18:59:34.109739 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:34.109716    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:34.114379 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:34.114359    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:34.989309 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:34.989278    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:34.990168 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:34.990151    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:36.232387 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.232352    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"]
Jul 09 18:59:36.235899 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.235881    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:36.238605 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.238576    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f5e743-serving-cert\""
Jul 09 18:59:36.248990 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.248968    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"]
Jul 09 18:59:36.319995 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.319971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-maas-api-tls\") pod \"maas-api-e2e-api-a-f5e743-678584c849-mwkxq\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:36.320124 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.320013    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkx7v\" (UniqueName: \"kubernetes.io/projected/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-kube-api-access-qkx7v\") pod \"maas-api-e2e-api-a-f5e743-678584c849-mwkxq\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:36.320124 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.320067    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-openshift-service-ca\") pod \"maas-api-e2e-api-a-f5e743-678584c849-mwkxq\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:36.421242 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.421216    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-openshift-service-ca\") pod \"maas-api-e2e-api-a-f5e743-678584c849-mwkxq\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:36.421359 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.421306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-maas-api-tls\") pod \"maas-api-e2e-api-a-f5e743-678584c849-mwkxq\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:36.421359 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.421331    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkx7v\" (UniqueName: \"kubernetes.io/projected/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-kube-api-access-qkx7v\") pod \"maas-api-e2e-api-a-f5e743-678584c849-mwkxq\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:36.421464 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:59:36.421448    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-serving-cert: secret "maas-api-e2e-api-a-f5e743-serving-cert" not found
Jul 09 18:59:36.421522 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:59:36.421512    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-maas-api-tls podName:499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:36.921493987 +0000 UTC m=+2054.057311437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-maas-api-tls") pod "maas-api-e2e-api-a-f5e743-678584c849-mwkxq" (UID: "499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4") : secret "maas-api-e2e-api-a-f5e743-serving-cert" not found
Jul 09 18:59:36.421889 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.421869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-openshift-service-ca\") pod \"maas-api-e2e-api-a-f5e743-678584c849-mwkxq\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:36.438755 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.438726    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkx7v\" (UniqueName: \"kubernetes.io/projected/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-kube-api-access-qkx7v\") pod \"maas-api-e2e-api-a-f5e743-678584c849-mwkxq\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:36.926928 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.926896    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-maas-api-tls\") pod \"maas-api-e2e-api-a-f5e743-678584c849-mwkxq\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:36.929153 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:36.929133    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-maas-api-tls\") pod \"maas-api-e2e-api-a-f5e743-678584c849-mwkxq\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:37.145969 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:37.145916    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:37.293196 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:37.293165    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"]
Jul 09 18:59:37.294787 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:59:37.294763    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod499f02ac_b2c3_4c11_ab74_f8ca1bf9d0a4.slice/crio-99f024856ecd6fb1616331d51e52fd1441364744b27e7b86b589e4768641e1e9 WatchSource:0}: Error finding container 99f024856ecd6fb1616331d51e52fd1441364744b27e7b86b589e4768641e1e9: Status 404 returned error can't find the container with id 99f024856ecd6fb1616331d51e52fd1441364744b27e7b86b589e4768641e1e9
Jul 09 18:59:38.004243 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:38.004214    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq" event={"ID":"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4","Type":"ContainerStarted","Data":"375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8"}
Jul 09 18:59:38.004444 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:38.004272    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:38.004444 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:38.004292    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq" event={"ID":"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4","Type":"ContainerStarted","Data":"99f024856ecd6fb1616331d51e52fd1441364744b27e7b86b589e4768641e1e9"}
Jul 09 18:59:38.026058 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:38.026015    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq" podStartSLOduration=1.5040426679999999 podStartE2EDuration="2.026001968s" totalImagesPullingTime="521.9593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:38.024075985 +0000 UTC m=+2055.159893456" watchObservedRunningTime="2026-07-09 18:59:38.026001968 +0000 UTC m=+2055.161819440"
Jul 09 18:59:44.014820 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.014790    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:44.713517 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.713485    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"]
Jul 09 18:59:44.717132 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.717113    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:44.720293 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.720274    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-6c0298-serving-cert\""
Jul 09 18:59:44.738389 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.738368    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"]
Jul 09 18:59:44.794799 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.794767    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/690563e8-eb95-4ad0-a10d-44a54e071e28-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:44.794925 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.794900    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8dnw\" (UniqueName: \"kubernetes.io/projected/690563e8-eb95-4ad0-a10d-44a54e071e28-kube-api-access-l8dnw\") pod \"maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:44.794978 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.794945    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/690563e8-eb95-4ad0-a10d-44a54e071e28-maas-api-tls\") pod \"maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:44.895546 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.895516    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8dnw\" (UniqueName: \"kubernetes.io/projected/690563e8-eb95-4ad0-a10d-44a54e071e28-kube-api-access-l8dnw\") pod \"maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:44.895748 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.895559    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/690563e8-eb95-4ad0-a10d-44a54e071e28-maas-api-tls\") pod \"maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:44.895748 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.895588    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/690563e8-eb95-4ad0-a10d-44a54e071e28-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:44.895748 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:59:44.895704    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-serving-cert: secret "maas-api-e2e-api-b-6c0298-serving-cert" not found
Jul 09 18:59:44.895935 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:59:44.895764    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/690563e8-eb95-4ad0-a10d-44a54e071e28-maas-api-tls podName:690563e8-eb95-4ad0-a10d-44a54e071e28 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:45.395748372 +0000 UTC m=+2062.531565822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/690563e8-eb95-4ad0-a10d-44a54e071e28-maas-api-tls") pod "maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg" (UID: "690563e8-eb95-4ad0-a10d-44a54e071e28") : secret "maas-api-e2e-api-b-6c0298-serving-cert" not found
Jul 09 18:59:44.896188 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.896171    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/690563e8-eb95-4ad0-a10d-44a54e071e28-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:44.908832 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:44.908810    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8dnw\" (UniqueName: \"kubernetes.io/projected/690563e8-eb95-4ad0-a10d-44a54e071e28-kube-api-access-l8dnw\") pod \"maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:45.400473 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:45.400445    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/690563e8-eb95-4ad0-a10d-44a54e071e28-maas-api-tls\") pod \"maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:45.402806 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:45.402782    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/690563e8-eb95-4ad0-a10d-44a54e071e28-maas-api-tls\") pod \"maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:45.627042 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:45.627015    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:45.825604 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:45.825580    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"]
Jul 09 18:59:45.827291 ip-10-0-135-28 kubenswrapper[2566]: W0709 18:59:45.827257    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690563e8_eb95_4ad0_a10d_44a54e071e28.slice/crio-c363b2b2f5ed1dceca620c45dac83ba7ebc5b0d1993228f684628b049b07898e WatchSource:0}: Error finding container c363b2b2f5ed1dceca620c45dac83ba7ebc5b0d1993228f684628b049b07898e: Status 404 returned error can't find the container with id c363b2b2f5ed1dceca620c45dac83ba7ebc5b0d1993228f684628b049b07898e
Jul 09 18:59:46.041494 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:46.041399    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg" event={"ID":"690563e8-eb95-4ad0-a10d-44a54e071e28","Type":"ContainerStarted","Data":"c363b2b2f5ed1dceca620c45dac83ba7ebc5b0d1993228f684628b049b07898e"}
Jul 09 18:59:47.048789 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:47.048761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg" event={"ID":"690563e8-eb95-4ad0-a10d-44a54e071e28","Type":"ContainerStarted","Data":"7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a"}
Jul 09 18:59:47.049219 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:47.048895    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:47.088516 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:47.088476    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg" podStartSLOduration=2.735352244 podStartE2EDuration="3.088463063s" totalImagesPullingTime="353.110819ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:47.088287921 +0000 UTC m=+2064.224105394" watchObservedRunningTime="2026-07-09 18:59:47.088463063 +0000 UTC m=+2064.224280534"
Jul 09 18:59:49.990469 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:49.990416    2566 patch_prober.go:28] interesting pod/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 09 18:59:49.990816 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:49.990471    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 09 18:59:50.064996 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:50.064940    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerID="d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061" exitCode=0
Jul 09 18:59:50.065090 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:50.065015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" event={"ID":"2a5a6723-1fec-4545-9f1a-7852bcdae2df","Type":"ContainerDied","Data":"d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061"}
Jul 09 18:59:50.065359 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:50.065347    2566 scope.go:117] "RemoveContainer" containerID="d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061"
Jul 09 18:59:50.066276 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:50.066251    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:59:50.066343 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:50.066328    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:59:50.066381 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:50.066366    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 18:59:51.072219 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:51.072180    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" event={"ID":"2a5a6723-1fec-4545-9f1a-7852bcdae2df","Type":"ContainerStarted","Data":"0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3"}
Jul 09 18:59:51.109438 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:51.109409    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:51.109621 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:51.109600    2566 patch_prober.go:28] interesting pod/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 09 18:59:51.109686 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:51.109663    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 09 18:59:52.109986 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:52.109955    2566 patch_prober.go:28] interesting pod/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 09 18:59:52.110366 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:52.110008    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 09 18:59:53.067846 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:53.067819    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 18:59:53.109393 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:53.109366    2566 patch_prober.go:28] interesting pod/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 09 18:59:53.109543 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:53.109420    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 09 18:59:54.109537 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:54.109506    2566 patch_prober.go:28] interesting pod/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 09 18:59:54.109944 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:54.109558    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 09 18:59:55.113560 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:55.113529    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:55.113933 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:55.113876    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:55.114506 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:55.114485    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 18:59:58.094745 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.094715    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"]
Jul 09 18:59:58.095178 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.094917    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq" podUID="499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4" containerName="maas-api" containerID="cri-o://375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8" gracePeriod=30
Jul 09 18:59:58.347858 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.347802    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:58.427631 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.427591    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkx7v\" (UniqueName: \"kubernetes.io/projected/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-kube-api-access-qkx7v\") pod \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") "
Jul 09 18:59:58.427821 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.427679    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-maas-api-tls\") pod \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") "
Jul 09 18:59:58.427821 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.427745    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-openshift-service-ca\") pod \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\" (UID: \"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4\") "
Jul 09 18:59:58.428090 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.428068    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4" (UID: "499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:59:58.429589 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.429569    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-kube-api-access-qkx7v" (OuterVolumeSpecName: "kube-api-access-qkx7v") pod "499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4" (UID: "499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4"). InnerVolumeSpecName "kube-api-access-qkx7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:59:58.429715 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.429696    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4" (UID: "499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:59:58.529220 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.529179    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-maas-api-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:59:58.529220 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.529215    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-openshift-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:59:58.529220 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:58.529226    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkx7v\" (UniqueName: \"kubernetes.io/projected/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4-kube-api-access-qkx7v\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 18:59:59.110819 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:59.110786    2566 generic.go:356] "Generic (PLEG): container finished" podID="499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4" containerID="375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8" exitCode=0
Jul 09 18:59:59.111197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:59.110830    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"
Jul 09 18:59:59.111197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:59.110867    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq" event={"ID":"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4","Type":"ContainerDied","Data":"375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8"}
Jul 09 18:59:59.111197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:59.110921    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq" event={"ID":"499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4","Type":"ContainerDied","Data":"99f024856ecd6fb1616331d51e52fd1441364744b27e7b86b589e4768641e1e9"}
Jul 09 18:59:59.111197 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:59.110944    2566 scope.go:117] "RemoveContainer" containerID="375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8"
Jul 09 18:59:59.121714 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:59.121695    2566 scope.go:117] "RemoveContainer" containerID="375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8"
Jul 09 18:59:59.121991 ip-10-0-135-28 kubenswrapper[2566]: E0709 18:59:59.121969    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8\": container with ID starting with 375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8 not found: ID does not exist" containerID="375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8"
Jul 09 18:59:59.122056 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:59.121999    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8"} err="failed to get container status \"375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8\": rpc error: code = NotFound desc = could not find container \"375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8\": container with ID starting with 375a3c0f1da4310e38e15bdfcdc5dc4e2355ed8594f9e5fea8f8fb7cf1a35de8 not found: ID does not exist"
Jul 09 18:59:59.137511 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:59.137489    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"]
Jul 09 18:59:59.142437 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:59.142418    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f5e743-678584c849-mwkxq"]
Jul 09 18:59:59.360264 ip-10-0-135-28 kubenswrapper[2566]: I0709 18:59:59.360231    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4" path="/var/lib/kubelet/pods/499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4/volumes"
Jul 09 19:00:04.428998 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:04.428966    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"]
Jul 09 19:00:04.429581 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:04.429163    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy" containerID="cri-o://0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3" gracePeriod=30
Jul 09 19:00:05.809699 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:05.809668    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"]
Jul 09 19:00:05.810161 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:05.809846    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg" podUID="690563e8-eb95-4ad0-a10d-44a54e071e28" containerName="maas-api" containerID="cri-o://7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a" gracePeriod=30
Jul 09 19:00:06.058247 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.058224    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 19:00:06.155349 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.155264    2566 generic.go:356] "Generic (PLEG): container finished" podID="690563e8-eb95-4ad0-a10d-44a54e071e28" containerID="7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a" exitCode=0
Jul 09 19:00:06.155349 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.155329    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"
Jul 09 19:00:06.155556 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.155344    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg" event={"ID":"690563e8-eb95-4ad0-a10d-44a54e071e28","Type":"ContainerDied","Data":"7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a"}
Jul 09 19:00:06.155556 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.155412    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg" event={"ID":"690563e8-eb95-4ad0-a10d-44a54e071e28","Type":"ContainerDied","Data":"c363b2b2f5ed1dceca620c45dac83ba7ebc5b0d1993228f684628b049b07898e"}
Jul 09 19:00:06.155556 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.155435    2566 scope.go:117] "RemoveContainer" containerID="7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a"
Jul 09 19:00:06.163307 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.163290    2566 scope.go:117] "RemoveContainer" containerID="7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a"
Jul 09 19:00:06.163569 ip-10-0-135-28 kubenswrapper[2566]: E0709 19:00:06.163550    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a\": container with ID starting with 7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a not found: ID does not exist" containerID="7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a"
Jul 09 19:00:06.163662 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.163580    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a"} err="failed to get container status \"7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a\": rpc error: code = NotFound desc = could not find container \"7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a\": container with ID starting with 7e77cceaba8e4348ed9096f9162aeceff1df924e5fd3bdfa3096dbf7052a8d6a not found: ID does not exist"
Jul 09 19:00:06.194292 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.194270    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8dnw\" (UniqueName: \"kubernetes.io/projected/690563e8-eb95-4ad0-a10d-44a54e071e28-kube-api-access-l8dnw\") pod \"690563e8-eb95-4ad0-a10d-44a54e071e28\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") "
Jul 09 19:00:06.194391 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.194309    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/690563e8-eb95-4ad0-a10d-44a54e071e28-openshift-service-ca\") pod \"690563e8-eb95-4ad0-a10d-44a54e071e28\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") "
Jul 09 19:00:06.194451 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.194434    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/690563e8-eb95-4ad0-a10d-44a54e071e28-maas-api-tls\") pod \"690563e8-eb95-4ad0-a10d-44a54e071e28\" (UID: \"690563e8-eb95-4ad0-a10d-44a54e071e28\") "
Jul 09 19:00:06.194660 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.194630    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/690563e8-eb95-4ad0-a10d-44a54e071e28-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "690563e8-eb95-4ad0-a10d-44a54e071e28" (UID: "690563e8-eb95-4ad0-a10d-44a54e071e28"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:00:06.194763 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.194724    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/690563e8-eb95-4ad0-a10d-44a54e071e28-openshift-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:06.196210 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.196187    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/690563e8-eb95-4ad0-a10d-44a54e071e28-kube-api-access-l8dnw" (OuterVolumeSpecName: "kube-api-access-l8dnw") pod "690563e8-eb95-4ad0-a10d-44a54e071e28" (UID: "690563e8-eb95-4ad0-a10d-44a54e071e28"). InnerVolumeSpecName "kube-api-access-l8dnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:00:06.196295 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.196265    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/690563e8-eb95-4ad0-a10d-44a54e071e28-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "690563e8-eb95-4ad0-a10d-44a54e071e28" (UID: "690563e8-eb95-4ad0-a10d-44a54e071e28"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:00:06.295592 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.295566    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/690563e8-eb95-4ad0-a10d-44a54e071e28-maas-api-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:06.295592 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.295588    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8dnw\" (UniqueName: \"kubernetes.io/projected/690563e8-eb95-4ad0-a10d-44a54e071e28-kube-api-access-l8dnw\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:06.477447 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.477416    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"]
Jul 09 19:00:06.482070 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:06.482042    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c0298-575d8dfcb4-ssnvg"]
Jul 09 19:00:07.358836 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:07.358802    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="690563e8-eb95-4ad0-a10d-44a54e071e28" path="/var/lib/kubelet/pods/690563e8-eb95-4ad0-a10d-44a54e071e28/volumes"
Jul 09 19:00:09.666899 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.666876    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 19:00:09.825848 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.825769    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mwx6\" (UniqueName: \"kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-kube-api-access-8mwx6\") pod \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") "
Jul 09 19:00:09.825848 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.825803    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-token\") pod \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") "
Jul 09 19:00:09.825848 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.825833    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-data\") pod \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") "
Jul 09 19:00:09.826124 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.825865    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-certs\") pod \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") "
Jul 09 19:00:09.826124 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.825894    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istiod-ca-cert\") pod \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") "
Jul 09 19:00:09.826124 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.825923    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-credential-socket\") pod \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") "
Jul 09 19:00:09.826124 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.825969    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-socket\") pod \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") "
Jul 09 19:00:09.826124 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.826013    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-podinfo\") pod \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") "
Jul 09 19:00:09.826124 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.826061    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-envoy\") pod \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\" (UID: \"2a5a6723-1fec-4545-9f1a-7852bcdae2df\") "
Jul 09 19:00:09.826438 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.826235    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2a5a6723-1fec-4545-9f1a-7852bcdae2df" (UID: "2a5a6723-1fec-4545-9f1a-7852bcdae2df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:00:09.826438 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.826350    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2a5a6723-1fec-4545-9f1a-7852bcdae2df" (UID: "2a5a6723-1fec-4545-9f1a-7852bcdae2df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:00:09.826438 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.826384    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2a5a6723-1fec-4545-9f1a-7852bcdae2df" (UID: "2a5a6723-1fec-4545-9f1a-7852bcdae2df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:00:09.826438 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.826425    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-credential-socket\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:09.826628 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.826456    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2a5a6723-1fec-4545-9f1a-7852bcdae2df" (UID: "2a5a6723-1fec-4545-9f1a-7852bcdae2df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:00:09.827004 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.826973    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-data" (OuterVolumeSpecName: "istio-data") pod "2a5a6723-1fec-4545-9f1a-7852bcdae2df" (UID: "2a5a6723-1fec-4545-9f1a-7852bcdae2df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:00:09.828159 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.828127    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-token" (OuterVolumeSpecName: "istio-token") pod "2a5a6723-1fec-4545-9f1a-7852bcdae2df" (UID: "2a5a6723-1fec-4545-9f1a-7852bcdae2df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:00:09.828258 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.828160    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-kube-api-access-8mwx6" (OuterVolumeSpecName: "kube-api-access-8mwx6") pod "2a5a6723-1fec-4545-9f1a-7852bcdae2df" (UID: "2a5a6723-1fec-4545-9f1a-7852bcdae2df"). InnerVolumeSpecName "kube-api-access-8mwx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:00:09.828258 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.828206    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2a5a6723-1fec-4545-9f1a-7852bcdae2df" (UID: "2a5a6723-1fec-4545-9f1a-7852bcdae2df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:00:09.828377 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.828358    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2a5a6723-1fec-4545-9f1a-7852bcdae2df" (UID: "2a5a6723-1fec-4545-9f1a-7852bcdae2df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 19:00:09.927253 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.927220    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mwx6\" (UniqueName: \"kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-kube-api-access-8mwx6\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:09.927253 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.927248    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-token\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:09.927253 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.927258    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-data\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:09.927465 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.927265    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-certs\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:09.927465 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.927273    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istiod-ca-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:09.927465 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.927281    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-workload-socket\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:09.927465 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.927288    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-podinfo\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:09.927465 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:09.927296    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a5a6723-1fec-4545-9f1a-7852bcdae2df-istio-envoy\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:00:10.178357 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.178326    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerID="0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3" exitCode=0
Jul 09 19:00:10.178534 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.178368    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" event={"ID":"2a5a6723-1fec-4545-9f1a-7852bcdae2df","Type":"ContainerDied","Data":"0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3"}
Jul 09 19:00:10.178534 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.178373    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"
Jul 09 19:00:10.178534 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.178409    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27" event={"ID":"2a5a6723-1fec-4545-9f1a-7852bcdae2df","Type":"ContainerDied","Data":"3828a9bfb782eb196b2e3d353574c15c3395e3bbb81b4fa502ee752d7cb34985"}
Jul 09 19:00:10.178534 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.178430    2566 scope.go:117] "RemoveContainer" containerID="0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3"
Jul 09 19:00:10.187343 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.187318    2566 scope.go:117] "RemoveContainer" containerID="d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061"
Jul 09 19:00:10.194947 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.194929    2566 scope.go:117] "RemoveContainer" containerID="0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3"
Jul 09 19:00:10.195171 ip-10-0-135-28 kubenswrapper[2566]: E0709 19:00:10.195151    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3\": container with ID starting with 0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3 not found: ID does not exist" containerID="0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3"
Jul 09 19:00:10.195223 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.195179    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3"} err="failed to get container status \"0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3\": rpc error: code = NotFound desc = could not find container \"0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3\": container with ID starting with 0838b40e41722db6283eb2da0936c3a193a0d2bc462188cdc023e44ebcedd0f3 not found: ID does not exist"
Jul 09 19:00:10.195223 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.195199    2566 scope.go:117] "RemoveContainer" containerID="d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061"
Jul 09 19:00:10.195398 ip-10-0-135-28 kubenswrapper[2566]: E0709 19:00:10.195387    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061\": container with ID starting with d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061 not found: ID does not exist" containerID="d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061"
Jul 09 19:00:10.195448 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.195404    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061"} err="failed to get container status \"d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061\": rpc error: code = NotFound desc = could not find container \"d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061\": container with ID starting with d5999fc11f07cbd7d6b14baf0b59717b30dcd28bdac6722158cacab7a0dcc061 not found: ID does not exist"
Jul 09 19:00:10.203836 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.203808    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"]
Jul 09 19:00:10.210924 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:10.210902    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f5e743-openshift-default-6659477bc-5qp27"]
Jul 09 19:00:11.359362 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:11.359326    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" path="/var/lib/kubelet/pods/2a5a6723-1fec-4545-9f1a-7852bcdae2df/volumes"
Jul 09 19:00:13.281540 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.281505    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"]
Jul 09 19:00:13.281923 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.281878    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4" containerName="maas-api"
Jul 09 19:00:13.281923 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.281890    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4" containerName="maas-api"
Jul 09 19:00:13.281923 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.281909    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy"
Jul 09 19:00:13.281923 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.281914    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy"
Jul 09 19:00:13.282058 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.281928    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy"
Jul 09 19:00:13.282058 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.281933    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy"
Jul 09 19:00:13.282058 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.281942    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="690563e8-eb95-4ad0-a10d-44a54e071e28" containerName="maas-api"
Jul 09 19:00:13.282058 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.281947    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="690563e8-eb95-4ad0-a10d-44a54e071e28" containerName="maas-api"
Jul 09 19:00:13.282058 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.282003    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="690563e8-eb95-4ad0-a10d-44a54e071e28" containerName="maas-api"
Jul 09 19:00:13.282058 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.282010    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy"
Jul 09 19:00:13.282058 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.282017    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="499f02ac-b2c3-4c11-ab74-f8ca1bf9d0a4" containerName="maas-api"
Jul 09 19:00:13.282058 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.282023    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a5a6723-1fec-4545-9f1a-7852bcdae2df" containerName="istio-proxy"
Jul 09 19:00:13.286277 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.286254    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.288737 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.288714    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-4805f3-openshift-default-dockercfg-4zx7z\""
Jul 09 19:00:13.301387 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.301365    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"]
Jul 09 19:00:13.457036 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.457002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-socket\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.457210 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.457044    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5853595-de01-4785-ac31-814f50370776-istiod-ca-cert\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.457210 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.457084    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-data\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.457210 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.457108    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vrsk\" (UniqueName: \"kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-kube-api-access-6vrsk\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.457210 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.457134    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-certs\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.457210 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.457152    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-credential-socket\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.457210 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.457168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-envoy\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.457210 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.457187    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-istio-token\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.457449 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.457263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5853595-de01-4785-ac31-814f50370776-istio-podinfo\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558341 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558273    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5853595-de01-4785-ac31-814f50370776-istiod-ca-cert\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558341 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558325    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-data\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558556 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558346    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vrsk\" (UniqueName: \"kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-kube-api-access-6vrsk\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558556 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558369    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-certs\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558556 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558387    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-credential-socket\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558556 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558410    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-envoy\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558556 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558427    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-istio-token\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558556 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558459    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5853595-de01-4785-ac31-814f50370776-istio-podinfo\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558556 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-socket\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558951 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558841    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-data\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558951 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558896    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-certs\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.558951 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558917    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-socket\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.559096 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.558970    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-credential-socket\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.559096 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.559030    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5853595-de01-4785-ac31-814f50370776-istiod-ca-cert\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.560958 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.560936    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-envoy\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.561068 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.561054    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5853595-de01-4785-ac31-814f50370776-istio-podinfo\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.571482 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.571463    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-istio-token\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.571562 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.571512    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vrsk\" (UniqueName: \"kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-kube-api-access-6vrsk\") pod \"e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") " pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.596196 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.596170    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:13.741134 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.741108    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"]
Jul 09 19:00:13.743265 ip-10-0-135-28 kubenswrapper[2566]: W0709 19:00:13.743236    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5853595_de01_4785_ac31_814f50370776.slice/crio-4e98097a9dc0b0047056616a32c0844347a3115ac9a869300f8ee9174c5b0a07 WatchSource:0}: Error finding container 4e98097a9dc0b0047056616a32c0844347a3115ac9a869300f8ee9174c5b0a07: Status 404 returned error can't find the container with id 4e98097a9dc0b0047056616a32c0844347a3115ac9a869300f8ee9174c5b0a07
Jul 09 19:00:13.745689 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.745607    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 19:00:13.745847 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.745704    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 19:00:13.745847 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:13.745751    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 19:00:14.201048 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:14.201011    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" event={"ID":"f5853595-de01-4785-ac31-814f50370776","Type":"ContainerStarted","Data":"914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4"}
Jul 09 19:00:14.201224 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:14.201061    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" event={"ID":"f5853595-de01-4785-ac31-814f50370776","Type":"ContainerStarted","Data":"4e98097a9dc0b0047056616a32c0844347a3115ac9a869300f8ee9174c5b0a07"}
Jul 09 19:00:14.229619 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:14.229573    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" podStartSLOduration=1.229559416 podStartE2EDuration="1.229559416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:00:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:14.227914256 +0000 UTC m=+2091.363731729" watchObservedRunningTime="2026-07-09 19:00:14.229559416 +0000 UTC m=+2091.365376887"
Jul 09 19:00:14.596498 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:14.596430    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:14.600980 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:14.600959    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:15.205807 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:15.205777    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:15.206725 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:15.206706    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:16.309052 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.309008    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"]
Jul 09 19:00:16.312546 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.312528    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.315671 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.315629    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-4805f3-serving-cert\""
Jul 09 19:00:16.332224 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.332204    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"]
Jul 09 19:00:16.483592 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.483563    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6spsw\" (UniqueName: \"kubernetes.io/projected/fee752cc-c495-4408-8fe6-d7ec62a59f53-kube-api-access-6spsw\") pod \"maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.483793 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.483699    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fee752cc-c495-4408-8fe6-d7ec62a59f53-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.483793 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.483768    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fee752cc-c495-4408-8fe6-d7ec62a59f53-maas-api-tls\") pod \"maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.584296 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.584220    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fee752cc-c495-4408-8fe6-d7ec62a59f53-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.584296 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.584260    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fee752cc-c495-4408-8fe6-d7ec62a59f53-maas-api-tls\") pod \"maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.584494 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.584300    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6spsw\" (UniqueName: \"kubernetes.io/projected/fee752cc-c495-4408-8fe6-d7ec62a59f53-kube-api-access-6spsw\") pod \"maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.584951 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.584932    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fee752cc-c495-4408-8fe6-d7ec62a59f53-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.586412 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.586391    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fee752cc-c495-4408-8fe6-d7ec62a59f53-maas-api-tls\") pod \"maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.597747 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.597728    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6spsw\" (UniqueName: \"kubernetes.io/projected/fee752cc-c495-4408-8fe6-d7ec62a59f53-kube-api-access-6spsw\") pod \"maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.623581 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.623560    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:16.778411 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:16.778378    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"]
Jul 09 19:00:16.781857 ip-10-0-135-28 kubenswrapper[2566]: W0709 19:00:16.781828    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfee752cc_c495_4408_8fe6_d7ec62a59f53.slice/crio-1177c12eb20ef7ac2705e5f6b7470f1046b7d3ea5a4e442e198d184ea32d12ca WatchSource:0}: Error finding container 1177c12eb20ef7ac2705e5f6b7470f1046b7d3ea5a4e442e198d184ea32d12ca: Status 404 returned error can't find the container with id 1177c12eb20ef7ac2705e5f6b7470f1046b7d3ea5a4e442e198d184ea32d12ca
Jul 09 19:00:17.219364 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:17.219327    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh" event={"ID":"fee752cc-c495-4408-8fe6-d7ec62a59f53","Type":"ContainerStarted","Data":"1177c12eb20ef7ac2705e5f6b7470f1046b7d3ea5a4e442e198d184ea32d12ca"}
Jul 09 19:00:18.226109 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:18.226067    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh" event={"ID":"fee752cc-c495-4408-8fe6-d7ec62a59f53","Type":"ContainerStarted","Data":"333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e"}
Jul 09 19:00:18.226559 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:18.226123    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:18.250140 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:18.250096    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh" podStartSLOduration=1.7973935330000002 podStartE2EDuration="2.250077616s" totalImagesPullingTime="452.684083ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:18.249581647 +0000 UTC m=+2095.385399119" watchObservedRunningTime="2026-07-09 19:00:18.250077616 +0000 UTC m=+2095.385895090"
Jul 09 19:00:24.236291 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:24.236245    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:00:25.190563 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.190532    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"]
Jul 09 19:00:25.195524 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.195507    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:25.198364 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.198341    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-172f88-serving-cert\""
Jul 09 19:00:25.211230 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.211207    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"]
Jul 09 19:00:25.271015 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.270978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhb7j\" (UniqueName: \"kubernetes.io/projected/faecd62e-21de-42d0-b094-1fc7f49bfd24-kube-api-access-bhb7j\") pod \"maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:25.271721 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.271054    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faecd62e-21de-42d0-b094-1fc7f49bfd24-openshift-service-ca\") pod \"maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:25.271721 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.271126    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faecd62e-21de-42d0-b094-1fc7f49bfd24-maas-api-tls\") pod \"maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:25.371807 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.371772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhb7j\" (UniqueName: \"kubernetes.io/projected/faecd62e-21de-42d0-b094-1fc7f49bfd24-kube-api-access-bhb7j\") pod \"maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:25.371985 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.371832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faecd62e-21de-42d0-b094-1fc7f49bfd24-openshift-service-ca\") pod \"maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:25.371985 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.371904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faecd62e-21de-42d0-b094-1fc7f49bfd24-maas-api-tls\") pod \"maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:25.372097 ip-10-0-135-28 kubenswrapper[2566]: E0709 19:00:25.372057    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-serving-cert: secret "maas-api-e2e-shared-b-172f88-serving-cert" not found
Jul 09 19:00:25.372155 ip-10-0-135-28 kubenswrapper[2566]: E0709 19:00:25.372126    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faecd62e-21de-42d0-b094-1fc7f49bfd24-maas-api-tls podName:faecd62e-21de-42d0-b094-1fc7f49bfd24 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:25.872104265 +0000 UTC m=+2103.007921718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/faecd62e-21de-42d0-b094-1fc7f49bfd24-maas-api-tls") pod "maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf" (UID: "faecd62e-21de-42d0-b094-1fc7f49bfd24") : secret "maas-api-e2e-shared-b-172f88-serving-cert" not found
Jul 09 19:00:25.372504 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.372483    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faecd62e-21de-42d0-b094-1fc7f49bfd24-openshift-service-ca\") pod \"maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:25.405466 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.405441    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhb7j\" (UniqueName: \"kubernetes.io/projected/faecd62e-21de-42d0-b094-1fc7f49bfd24-kube-api-access-bhb7j\") pod \"maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:25.877241 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.877204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faecd62e-21de-42d0-b094-1fc7f49bfd24-maas-api-tls\") pod \"maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:25.879717 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:25.879691    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faecd62e-21de-42d0-b094-1fc7f49bfd24-maas-api-tls\") pod \"maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:26.105818 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:26.105745    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:26.294854 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:26.294830    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"]
Jul 09 19:00:26.297313 ip-10-0-135-28 kubenswrapper[2566]: W0709 19:00:26.297277    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaecd62e_21de_42d0_b094_1fc7f49bfd24.slice/crio-07c824bc9f3d50bf3c782ad25d45af6c5fb57230964b5779ee2cfe240006ca37 WatchSource:0}: Error finding container 07c824bc9f3d50bf3c782ad25d45af6c5fb57230964b5779ee2cfe240006ca37: Status 404 returned error can't find the container with id 07c824bc9f3d50bf3c782ad25d45af6c5fb57230964b5779ee2cfe240006ca37
Jul 09 19:00:27.283017 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:27.282979    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf" event={"ID":"faecd62e-21de-42d0-b094-1fc7f49bfd24","Type":"ContainerStarted","Data":"07c824bc9f3d50bf3c782ad25d45af6c5fb57230964b5779ee2cfe240006ca37"}
Jul 09 19:00:28.289885 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:28.289849    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf" event={"ID":"faecd62e-21de-42d0-b094-1fc7f49bfd24","Type":"ContainerStarted","Data":"4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f"}
Jul 09 19:00:28.290310 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:28.289911    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:28.310478 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:28.310427    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf" podStartSLOduration=2.154601839 podStartE2EDuration="3.310410284s" totalImagesPullingTime="1.155808445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:28.308765548 +0000 UTC m=+2105.444583022" watchObservedRunningTime="2026-07-09 19:00:28.310410284 +0000 UTC m=+2105.446227756"
Jul 09 19:00:30.207320 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:30.207275    2566 patch_prober.go:28] interesting pod/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 09 19:00:30.207734 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:30.207354    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 09 19:00:30.301024 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:30.300994    2566 generic.go:356] "Generic (PLEG): container finished" podID="f5853595-de01-4785-ac31-814f50370776" containerID="914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4" exitCode=0
Jul 09 19:00:30.301159 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:30.301069    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" event={"ID":"f5853595-de01-4785-ac31-814f50370776","Type":"ContainerDied","Data":"914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4"}
Jul 09 19:00:30.301525 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:30.301507    2566 scope.go:117] "RemoveContainer" containerID="914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4"
Jul 09 19:00:30.302513 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:30.302487    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 19:00:30.302602 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:30.302577    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 19:00:30.302702 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:30.302683    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 19:00:31.307793 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:31.307758    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" event={"ID":"f5853595-de01-4785-ac31-814f50370776","Type":"ContainerStarted","Data":"53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d"}
Jul 09 19:00:31.597437 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:31.597345    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:31.597579 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:31.597549    2566 patch_prober.go:28] interesting pod/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 09 19:00:31.597621 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:31.597598    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 09 19:00:32.596932 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:32.596899    2566 patch_prober.go:28] interesting pod/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 09 19:00:32.597331 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:32.596948    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 09 19:00:33.596965 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:33.596928    2566 patch_prober.go:28] interesting pod/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 09 19:00:33.597435 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:33.596989    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 09 19:00:34.301090 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:34.301053    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:00:34.597410 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:34.597332    2566 patch_prober.go:28] interesting pod/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 09 19:00:34.597410 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:34.597381    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 09 19:00:35.600579 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:35.600541    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:35.601061 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:35.600876    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:00:35.601445 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:00:35.601426    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:07:02.279142 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.279106    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"]
Jul 09 19:07:02.279736 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.279330    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh" podUID="fee752cc-c495-4408-8fe6-d7ec62a59f53" containerName="maas-api" containerID="cri-o://333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e" gracePeriod=30
Jul 09 19:07:02.531408 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.531346    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:07:02.607493 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.607465    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fee752cc-c495-4408-8fe6-d7ec62a59f53-openshift-service-ca\") pod \"fee752cc-c495-4408-8fe6-d7ec62a59f53\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") "
Jul 09 19:07:02.607493 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.607495    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6spsw\" (UniqueName: \"kubernetes.io/projected/fee752cc-c495-4408-8fe6-d7ec62a59f53-kube-api-access-6spsw\") pod \"fee752cc-c495-4408-8fe6-d7ec62a59f53\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") "
Jul 09 19:07:02.607680 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.607544    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fee752cc-c495-4408-8fe6-d7ec62a59f53-maas-api-tls\") pod \"fee752cc-c495-4408-8fe6-d7ec62a59f53\" (UID: \"fee752cc-c495-4408-8fe6-d7ec62a59f53\") "
Jul 09 19:07:02.607904 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.607877    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fee752cc-c495-4408-8fe6-d7ec62a59f53-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fee752cc-c495-4408-8fe6-d7ec62a59f53" (UID: "fee752cc-c495-4408-8fe6-d7ec62a59f53"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:07:02.609586 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.609553    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fee752cc-c495-4408-8fe6-d7ec62a59f53-kube-api-access-6spsw" (OuterVolumeSpecName: "kube-api-access-6spsw") pod "fee752cc-c495-4408-8fe6-d7ec62a59f53" (UID: "fee752cc-c495-4408-8fe6-d7ec62a59f53"). InnerVolumeSpecName "kube-api-access-6spsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:07:02.609586 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.609573    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fee752cc-c495-4408-8fe6-d7ec62a59f53-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fee752cc-c495-4408-8fe6-d7ec62a59f53" (UID: "fee752cc-c495-4408-8fe6-d7ec62a59f53"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:07:02.708393 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.708370    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fee752cc-c495-4408-8fe6-d7ec62a59f53-openshift-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:02.708393 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.708391    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6spsw\" (UniqueName: \"kubernetes.io/projected/fee752cc-c495-4408-8fe6-d7ec62a59f53-kube-api-access-6spsw\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:02.708527 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:02.708401    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fee752cc-c495-4408-8fe6-d7ec62a59f53-maas-api-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:03.223572 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:03.223545    2566 generic.go:356] "Generic (PLEG): container finished" podID="fee752cc-c495-4408-8fe6-d7ec62a59f53" containerID="333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e" exitCode=0
Jul 09 19:07:03.223753 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:03.223588    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"
Jul 09 19:07:03.223753 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:03.223636    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh" event={"ID":"fee752cc-c495-4408-8fe6-d7ec62a59f53","Type":"ContainerDied","Data":"333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e"}
Jul 09 19:07:03.223753 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:03.223702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh" event={"ID":"fee752cc-c495-4408-8fe6-d7ec62a59f53","Type":"ContainerDied","Data":"1177c12eb20ef7ac2705e5f6b7470f1046b7d3ea5a4e442e198d184ea32d12ca"}
Jul 09 19:07:03.223753 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:03.223729    2566 scope.go:117] "RemoveContainer" containerID="333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e"
Jul 09 19:07:03.233420 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:03.233400    2566 scope.go:117] "RemoveContainer" containerID="333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e"
Jul 09 19:07:03.233689 ip-10-0-135-28 kubenswrapper[2566]: E0709 19:07:03.233670    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e\": container with ID starting with 333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e not found: ID does not exist" containerID="333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e"
Jul 09 19:07:03.233745 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:03.233697    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e"} err="failed to get container status \"333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e\": rpc error: code = NotFound desc = could not find container \"333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e\": container with ID starting with 333e60c367864a0d4a14d5edf0d22394c5e6b58dd5e9909b495c3a64422da99e not found: ID does not exist"
Jul 09 19:07:03.245585 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:03.245563    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"]
Jul 09 19:07:03.249234 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:03.249209    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4805f3-6f977bc7bd-h6pxh"]
Jul 09 19:07:03.359196 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:03.359164    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fee752cc-c495-4408-8fe6-d7ec62a59f53" path="/var/lib/kubelet/pods/fee752cc-c495-4408-8fe6-d7ec62a59f53/volumes"
Jul 09 19:07:09.030401 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:09.030370    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"]
Jul 09 19:07:09.030928 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:09.030561    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy" containerID="cri-o://53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d" gracePeriod=30
Jul 09 19:07:10.280182 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.280147    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"]
Jul 09 19:07:10.280601 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.280336    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf" podUID="faecd62e-21de-42d0-b094-1fc7f49bfd24" containerName="maas-api" containerID="cri-o://4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f" gracePeriod=30
Jul 09 19:07:10.524291 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.524260    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:07:10.676970 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.676945    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faecd62e-21de-42d0-b094-1fc7f49bfd24-openshift-service-ca\") pod \"faecd62e-21de-42d0-b094-1fc7f49bfd24\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") "
Jul 09 19:07:10.677106 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.677051    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faecd62e-21de-42d0-b094-1fc7f49bfd24-maas-api-tls\") pod \"faecd62e-21de-42d0-b094-1fc7f49bfd24\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") "
Jul 09 19:07:10.677106 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.677089    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhb7j\" (UniqueName: \"kubernetes.io/projected/faecd62e-21de-42d0-b094-1fc7f49bfd24-kube-api-access-bhb7j\") pod \"faecd62e-21de-42d0-b094-1fc7f49bfd24\" (UID: \"faecd62e-21de-42d0-b094-1fc7f49bfd24\") "
Jul 09 19:07:10.677301 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.677278    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faecd62e-21de-42d0-b094-1fc7f49bfd24-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "faecd62e-21de-42d0-b094-1fc7f49bfd24" (UID: "faecd62e-21de-42d0-b094-1fc7f49bfd24"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:07:10.677405 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.677393    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faecd62e-21de-42d0-b094-1fc7f49bfd24-openshift-service-ca\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:10.678927 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.678904    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faecd62e-21de-42d0-b094-1fc7f49bfd24-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "faecd62e-21de-42d0-b094-1fc7f49bfd24" (UID: "faecd62e-21de-42d0-b094-1fc7f49bfd24"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:07:10.679036 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.678997    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faecd62e-21de-42d0-b094-1fc7f49bfd24-kube-api-access-bhb7j" (OuterVolumeSpecName: "kube-api-access-bhb7j") pod "faecd62e-21de-42d0-b094-1fc7f49bfd24" (UID: "faecd62e-21de-42d0-b094-1fc7f49bfd24"). InnerVolumeSpecName "kube-api-access-bhb7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:07:10.778101 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.778077    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faecd62e-21de-42d0-b094-1fc7f49bfd24-maas-api-tls\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:10.778101 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:10.778097    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhb7j\" (UniqueName: \"kubernetes.io/projected/faecd62e-21de-42d0-b094-1fc7f49bfd24-kube-api-access-bhb7j\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:11.272215 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:11.272171    2566 generic.go:356] "Generic (PLEG): container finished" podID="faecd62e-21de-42d0-b094-1fc7f49bfd24" containerID="4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f" exitCode=0
Jul 09 19:07:11.272403 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:11.272233    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"
Jul 09 19:07:11.272403 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:11.272251    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf" event={"ID":"faecd62e-21de-42d0-b094-1fc7f49bfd24","Type":"ContainerDied","Data":"4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f"}
Jul 09 19:07:11.272403 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:11.272340    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf" event={"ID":"faecd62e-21de-42d0-b094-1fc7f49bfd24","Type":"ContainerDied","Data":"07c824bc9f3d50bf3c782ad25d45af6c5fb57230964b5779ee2cfe240006ca37"}
Jul 09 19:07:11.272403 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:11.272376    2566 scope.go:117] "RemoveContainer" containerID="4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f"
Jul 09 19:07:11.282949 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:11.282746    2566 scope.go:117] "RemoveContainer" containerID="4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f"
Jul 09 19:07:11.283194 ip-10-0-135-28 kubenswrapper[2566]: E0709 19:07:11.282994    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f\": container with ID starting with 4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f not found: ID does not exist" containerID="4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f"
Jul 09 19:07:11.283194 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:11.283035    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f"} err="failed to get container status \"4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f\": rpc error: code = NotFound desc = could not find container \"4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f\": container with ID starting with 4ea24f6dbfc504a8740d8dce33729a3569fb330dd7c7b197f1f72a30805e148f not found: ID does not exist"
Jul 09 19:07:11.299025 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:11.299004    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"]
Jul 09 19:07:11.303777 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:11.303755    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-172f88-689f8fb5c7-dzmdf"]
Jul 09 19:07:11.358676 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:11.358634    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faecd62e-21de-42d0-b094-1fc7f49bfd24" path="/var/lib/kubelet/pods/faecd62e-21de-42d0-b094-1fc7f49bfd24/volumes"
Jul 09 19:07:14.280345 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.280322    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:07:14.289597 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.289543    2566 generic.go:356] "Generic (PLEG): container finished" podID="f5853595-de01-4785-ac31-814f50370776" containerID="53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d" exitCode=0
Jul 09 19:07:14.289597 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.289578    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"
Jul 09 19:07:14.289744 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.289620    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" event={"ID":"f5853595-de01-4785-ac31-814f50370776","Type":"ContainerDied","Data":"53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d"}
Jul 09 19:07:14.289744 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.289685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw" event={"ID":"f5853595-de01-4785-ac31-814f50370776","Type":"ContainerDied","Data":"4e98097a9dc0b0047056616a32c0844347a3115ac9a869300f8ee9174c5b0a07"}
Jul 09 19:07:14.289744 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.289709    2566 scope.go:117] "RemoveContainer" containerID="53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d"
Jul 09 19:07:14.297624 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.297606    2566 scope.go:117] "RemoveContainer" containerID="914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4"
Jul 09 19:07:14.306812 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.306791    2566 scope.go:117] "RemoveContainer" containerID="53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d"
Jul 09 19:07:14.307207 ip-10-0-135-28 kubenswrapper[2566]: E0709 19:07:14.307071    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d\": container with ID starting with 53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d not found: ID does not exist" containerID="53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d"
Jul 09 19:07:14.307207 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.307111    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d"} err="failed to get container status \"53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d\": rpc error: code = NotFound desc = could not find container \"53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d\": container with ID starting with 53325fd3ebfdd2aefc018450ffc010beaf87ad37922e406a5006cd22f91a158d not found: ID does not exist"
Jul 09 19:07:14.307207 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.307134    2566 scope.go:117] "RemoveContainer" containerID="914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4"
Jul 09 19:07:14.307503 ip-10-0-135-28 kubenswrapper[2566]: E0709 19:07:14.307475    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4\": container with ID starting with 914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4 not found: ID does not exist" containerID="914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4"
Jul 09 19:07:14.307612 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.307508    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4"} err="failed to get container status \"914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4\": rpc error: code = NotFound desc = could not find container \"914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4\": container with ID starting with 914f9d14ef6d9187d990ff36999614c37788cf92175398d0b5d78825f70384e4 not found: ID does not exist"
Jul 09 19:07:14.405504 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405483    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-credential-socket\") pod \"f5853595-de01-4785-ac31-814f50370776\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") "
Jul 09 19:07:14.405599 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405513    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5853595-de01-4785-ac31-814f50370776-istio-podinfo\") pod \"f5853595-de01-4785-ac31-814f50370776\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") "
Jul 09 19:07:14.405599 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405537    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5853595-de01-4785-ac31-814f50370776-istiod-ca-cert\") pod \"f5853595-de01-4785-ac31-814f50370776\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") "
Jul 09 19:07:14.405599 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405587    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-data\") pod \"f5853595-de01-4785-ac31-814f50370776\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") "
Jul 09 19:07:14.405777 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405606    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-socket\") pod \"f5853595-de01-4785-ac31-814f50370776\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") "
Jul 09 19:07:14.405777 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405625    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vrsk\" (UniqueName: \"kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-kube-api-access-6vrsk\") pod \"f5853595-de01-4785-ac31-814f50370776\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") "
Jul 09 19:07:14.405777 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405682    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-istio-token\") pod \"f5853595-de01-4785-ac31-814f50370776\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") "
Jul 09 19:07:14.405777 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405708    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-envoy\") pod \"f5853595-de01-4785-ac31-814f50370776\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") "
Jul 09 19:07:14.405777 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405728    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-certs\") pod \"f5853595-de01-4785-ac31-814f50370776\" (UID: \"f5853595-de01-4785-ac31-814f50370776\") "
Jul 09 19:07:14.406022 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405792    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f5853595-de01-4785-ac31-814f50370776" (UID: "f5853595-de01-4785-ac31-814f50370776"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:07:14.406022 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.405912    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f5853595-de01-4785-ac31-814f50370776" (UID: "f5853595-de01-4785-ac31-814f50370776"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:07:14.406126 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.406019    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5853595-de01-4785-ac31-814f50370776-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f5853595-de01-4785-ac31-814f50370776" (UID: "f5853595-de01-4785-ac31-814f50370776"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:07:14.406126 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.406035    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-credential-socket\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:14.406126 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.406086    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-socket\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:14.406230 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.406117    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f5853595-de01-4785-ac31-814f50370776" (UID: "f5853595-de01-4785-ac31-814f50370776"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:07:14.406539 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.406512    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-data" (OuterVolumeSpecName: "istio-data") pod "f5853595-de01-4785-ac31-814f50370776" (UID: "f5853595-de01-4785-ac31-814f50370776"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:07:14.407901 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.407879    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f5853595-de01-4785-ac31-814f50370776" (UID: "f5853595-de01-4785-ac31-814f50370776"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:07:14.408002 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.407897    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-kube-api-access-6vrsk" (OuterVolumeSpecName: "kube-api-access-6vrsk") pod "f5853595-de01-4785-ac31-814f50370776" (UID: "f5853595-de01-4785-ac31-814f50370776"). InnerVolumeSpecName "kube-api-access-6vrsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:07:14.408068 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.408003    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5853595-de01-4785-ac31-814f50370776-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f5853595-de01-4785-ac31-814f50370776" (UID: "f5853595-de01-4785-ac31-814f50370776"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 19:07:14.408068 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.408034    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5853595-de01-4785-ac31-814f50370776" (UID: "f5853595-de01-4785-ac31-814f50370776"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:07:14.506499 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.506469    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-data\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:14.506499 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.506496    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vrsk\" (UniqueName: \"kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-kube-api-access-6vrsk\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:14.506641 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.506512    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5853595-de01-4785-ac31-814f50370776-istio-token\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:14.506641 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.506524    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-istio-envoy\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:14.506641 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.506535    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5853595-de01-4785-ac31-814f50370776-workload-certs\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:14.506641 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.506545    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5853595-de01-4785-ac31-814f50370776-istio-podinfo\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:14.506641 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.506557    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5853595-de01-4785-ac31-814f50370776-istiod-ca-cert\") on node \"ip-10-0-135-28.ec2.internal\" DevicePath \"\""
Jul 09 19:07:14.610947 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.610918    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"]
Jul 09 19:07:14.617114 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:14.617091    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-4805f3-openshift-default-5f6c8bb567-zlxrw"]
Jul 09 19:07:15.359374 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:15.359290    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5853595-de01-4785-ac31-814f50370776" path="/var/lib/kubelet/pods/f5853595-de01-4785-ac31-814f50370776/volumes"
Jul 09 19:07:22.291160 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:22.291128    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5797656c88-hzbk8_e3260026-8158-4ccf-b8c0-203e39775fa1/manager/0.log"
Jul 09 19:07:22.638867 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:22.638789    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f7885b79-hv86z_08d0246d-eac4-4424-b007-6ee9d2a34f1c/copy-manifests/0.log"
Jul 09 19:07:22.650311 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:22.650277    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f7885b79-hv86z_08d0246d-eac4-4424-b007-6ee9d2a34f1c/manager/0.log"
Jul 09 19:07:23.258020 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:23.257985    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-794d8cb9fb-9wqnt_40a9b75d-475b-40d7-859e-b0544b531c7d/maas-api/0.log"
Jul 09 19:07:23.366089 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:23.366062    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-s5bmd_f83ae15b-25a1-489e-ac7d-c5c1b259d594/postgres/0.log"
Jul 09 19:07:25.887974 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:25.887940    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rn4db_7a966206-99c8-4a39-968f-e92a3d860cdd/discovery/0.log"
Jul 09 19:07:26.221343 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:26.221316    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7dqvt_03aabde7-0427-4b32-afe1-dca90a34ae24/istio-proxy/1.log"
Jul 09 19:07:26.466834 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:26.466806    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-569l6_8032829c-1601-4456-beea-406d065a820e/payload-processing/0.log"
Jul 09 19:07:27.111154 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:27.111116    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jb8bq_6f4df0c8-5427-4174-acaf-80e59edd50d0/main/0.log"
Jul 09 19:07:27.220184 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:27.220154    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhknnx_46c70108-50bc-457e-bdb6-1556e17aefe1/main/0.log"
Jul 09 19:07:33.840437 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:33.840405    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7m46n_9876487f-ee1d-4875-940e-d0032bc17427/global-pull-secret-syncer/0.log"
Jul 09 19:07:34.043072 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:34.043048    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9wx8q_2bb32097-7550-49c6-86fa-5d8248ace642/konnectivity-agent/0.log"
Jul 09 19:07:34.109580 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:34.109519    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-28.ec2.internal_e502bf3c1df2bd74d5cf28e4fde27161/haproxy/0.log"
Jul 09 19:07:40.606900 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:40.606874    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_593b26cd-bff8-426a-9d28-0cee362766b8/alertmanager/0.log"
Jul 09 19:07:40.629354 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:40.629328    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_593b26cd-bff8-426a-9d28-0cee362766b8/config-reloader/0.log"
Jul 09 19:07:40.650935 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:40.650910    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_593b26cd-bff8-426a-9d28-0cee362766b8/kube-rbac-proxy-web/0.log"
Jul 09 19:07:40.673924 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:40.673901    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_593b26cd-bff8-426a-9d28-0cee362766b8/kube-rbac-proxy/0.log"
Jul 09 19:07:40.699031 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:40.699014    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_593b26cd-bff8-426a-9d28-0cee362766b8/kube-rbac-proxy-metric/0.log"
Jul 09 19:07:40.719949 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:40.719930    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_593b26cd-bff8-426a-9d28-0cee362766b8/prom-label-proxy/0.log"
Jul 09 19:07:40.741445 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:40.741425    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_593b26cd-bff8-426a-9d28-0cee362766b8/init-config-reloader/0.log"
Jul 09 19:07:40.787301 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:40.787272    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-zkbpj_87da33a9-c88f-4f09-ac99-9bc91c35d81f/cluster-monitoring-operator/0.log"
Jul 09 19:07:40.990389 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:40.990370    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-76jqh_f10348ed-d7cb-4ea4-a599-ea5613e9abb7/node-exporter/0.log"
Jul 09 19:07:41.014477 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.014449    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-76jqh_f10348ed-d7cb-4ea4-a599-ea5613e9abb7/kube-rbac-proxy/0.log"
Jul 09 19:07:41.036313 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.036296    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-76jqh_f10348ed-d7cb-4ea4-a599-ea5613e9abb7/init-textfile/0.log"
Jul 09 19:07:41.266533 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.266510    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-ctdf7_f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa/kube-rbac-proxy-main/0.log"
Jul 09 19:07:41.295449 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.295426    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-ctdf7_f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa/kube-rbac-proxy-self/0.log"
Jul 09 19:07:41.325948 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.325929    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-ctdf7_f1a4b2f7-81a2-4d82-9944-f0c0e01fbffa/openshift-state-metrics/0.log"
Jul 09 19:07:41.375261 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.375238    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70b3e53a-50ef-4766-a5df-929d0c0b0620/prometheus/0.log"
Jul 09 19:07:41.399422 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.399403    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70b3e53a-50ef-4766-a5df-929d0c0b0620/config-reloader/0.log"
Jul 09 19:07:41.419805 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.419781    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70b3e53a-50ef-4766-a5df-929d0c0b0620/thanos-sidecar/0.log"
Jul 09 19:07:41.444701 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.444627    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70b3e53a-50ef-4766-a5df-929d0c0b0620/kube-rbac-proxy-web/0.log"
Jul 09 19:07:41.466752 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.466727    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70b3e53a-50ef-4766-a5df-929d0c0b0620/kube-rbac-proxy/0.log"
Jul 09 19:07:41.487298 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.487277    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70b3e53a-50ef-4766-a5df-929d0c0b0620/kube-rbac-proxy-thanos/0.log"
Jul 09 19:07:41.509993 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.509969    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70b3e53a-50ef-4766-a5df-929d0c0b0620/init-config-reloader/0.log"
Jul 09 19:07:41.611512 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:41.611490    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-v9wdw_a0d2a938-0ca6-4a72-94f8-3a6b9b1f1f63/prometheus-operator-admission-webhook/0.log"
Jul 09 19:07:43.879529 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879491    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"]
Jul 09 19:07:43.879922 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879862    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fee752cc-c495-4408-8fe6-d7ec62a59f53" containerName="maas-api"
Jul 09 19:07:43.879922 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879873    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fee752cc-c495-4408-8fe6-d7ec62a59f53" containerName="maas-api"
Jul 09 19:07:43.879922 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879886    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy"
Jul 09 19:07:43.879922 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879891    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy"
Jul 09 19:07:43.879922 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879898    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faecd62e-21de-42d0-b094-1fc7f49bfd24" containerName="maas-api"
Jul 09 19:07:43.879922 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879904    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="faecd62e-21de-42d0-b094-1fc7f49bfd24" containerName="maas-api"
Jul 09 19:07:43.879922 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879911    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy"
Jul 09 19:07:43.879922 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879917    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy"
Jul 09 19:07:43.880168 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879979    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy"
Jul 09 19:07:43.880168 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879987    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="faecd62e-21de-42d0-b094-1fc7f49bfd24" containerName="maas-api"
Jul 09 19:07:43.880168 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.879995    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5853595-de01-4785-ac31-814f50370776" containerName="istio-proxy"
Jul 09 19:07:43.880168 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.880002    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fee752cc-c495-4408-8fe6-d7ec62a59f53" containerName="maas-api"
Jul 09 19:07:43.883330 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.883307    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:43.885772 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.885746    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9jbd2\"/\"kube-root-ca.crt\""
Jul 09 19:07:43.886544 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.886528    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9jbd2\"/\"openshift-service-ca.crt\""
Jul 09 19:07:43.886630 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.886534    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9jbd2\"/\"default-dockercfg-pg4rv\""
Jul 09 19:07:43.894129 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.894095    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"]
Jul 09 19:07:43.945538 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.945519    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-proc\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:43.945683 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.945549    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr7s9\" (UniqueName: \"kubernetes.io/projected/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-kube-api-access-dr7s9\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:43.945683 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.945568    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-podres\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:43.945683 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.945589    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-lib-modules\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:43.945683 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:43.945603    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-sys\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.046849 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.046827    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-proc\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.046953 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.046860    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr7s9\" (UniqueName: \"kubernetes.io/projected/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-kube-api-access-dr7s9\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.046953 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.046878    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-podres\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.046953 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.046902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-lib-modules\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.046953 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.046937    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-sys\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.046953 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.046944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-proc\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.047145 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.047032    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-sys\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.047145 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.047051    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-podres\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.047145 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.047074    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-lib-modules\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.055825 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.055805    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr7s9\" (UniqueName: \"kubernetes.io/projected/27865ec1-d1ad-4dbc-a721-4de2b1bf6a27-kube-api-access-dr7s9\") pod \"perf-node-gather-daemonset-7nlwr\" (UID: \"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27\") " pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.194349 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.194328    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:44.323740 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.323708    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"]
Jul 09 19:07:44.326906 ip-10-0-135-28 kubenswrapper[2566]: W0709 19:07:44.326879    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod27865ec1_d1ad_4dbc_a721_4de2b1bf6a27.slice/crio-b31d628f731df8d8fee2085702a550f16d795a28acf5c67866841eadc1a8c6a2 WatchSource:0}: Error finding container b31d628f731df8d8fee2085702a550f16d795a28acf5c67866841eadc1a8c6a2: Status 404 returned error can't find the container with id b31d628f731df8d8fee2085702a550f16d795a28acf5c67866841eadc1a8c6a2
Jul 09 19:07:44.431034 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.431004    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr" event={"ID":"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27","Type":"ContainerStarted","Data":"b31d628f731df8d8fee2085702a550f16d795a28acf5c67866841eadc1a8c6a2"}
Jul 09 19:07:44.448077 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:44.448025    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-mw59t_54a5b984-6393-45da-8d25-0d86810f0c18/download-server/0.log"
Jul 09 19:07:45.438196 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:45.438164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr" event={"ID":"27865ec1-d1ad-4dbc-a721-4de2b1bf6a27","Type":"ContainerStarted","Data":"700a7a6d4e7ff2948c48836e889693e4c4909c06d9bba3a0194d9c4a22aa578c"}
Jul 09 19:07:45.438593 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:45.438286    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:45.454550 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:45.454494    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr" podStartSLOduration=2.4544810679999998 podStartE2EDuration="2.454481068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:45.454188002 +0000 UTC m=+2542.590005468" watchObservedRunningTime="2026-07-09 19:07:45.454481068 +0000 UTC m=+2542.590298540"
Jul 09 19:07:46.392142 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:46.392088    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xv799_6f37ae45-4062-40db-9fc7-6fcec23b742e/dns/0.log"
Jul 09 19:07:46.439228 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:46.439200    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xv799_6f37ae45-4062-40db-9fc7-6fcec23b742e/kube-rbac-proxy/0.log"
Jul 09 19:07:46.466780 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:46.466755    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2mb6q_59b49275-d607-420f-acaf-10d3b9843b44/dns-node-resolver/0.log"
Jul 09 19:07:47.155269 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:47.155243    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-slptj_07a65c98-48c0-4cd8-a737-5908696e29b6/node-ca/0.log"
Jul 09 19:07:48.331487 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:48.331461    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rn4db_7a966206-99c8-4a39-968f-e92a3d860cdd/discovery/0.log"
Jul 09 19:07:48.415976 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:48.415954    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7dqvt_03aabde7-0427-4b32-afe1-dca90a34ae24/istio-proxy/0.log"
Jul 09 19:07:48.482297 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:48.482243    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7dqvt_03aabde7-0427-4b32-afe1-dca90a34ae24/istio-proxy/1.log"
Jul 09 19:07:48.563737 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:48.563706    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-569l6_8032829c-1601-4456-beea-406d065a820e/payload-processing/0.log"
Jul 09 19:07:49.347497 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:49.347454    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xdlwn_274a4b22-0f83-4378-a747-f6a741e5abf3/serve-healthcheck-canary/0.log"
Jul 09 19:07:49.870933 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:49.870896    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-ctcrt_ee7722b6-b03d-4160-89d5-a7aaad6d122e/insights-operator/0.log"
Jul 09 19:07:49.872121 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:49.872094    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-ctcrt_ee7722b6-b03d-4160-89d5-a7aaad6d122e/insights-operator/1.log"
Jul 09 19:07:49.894777 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:49.894752    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c5x6g_a04edb26-2f53-47f1-b1cd-b94142d242dd/kube-rbac-proxy/0.log"
Jul 09 19:07:49.918506 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:49.918478    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c5x6g_a04edb26-2f53-47f1-b1cd-b94142d242dd/exporter/0.log"
Jul 09 19:07:49.945098 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:49.945080    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c5x6g_a04edb26-2f53-47f1-b1cd-b94142d242dd/extractor/0.log"
Jul 09 19:07:51.452818 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:51.452789    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9jbd2/perf-node-gather-daemonset-7nlwr"
Jul 09 19:07:52.803753 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:52.803721    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5797656c88-hzbk8_e3260026-8158-4ccf-b8c0-203e39775fa1/manager/0.log"
Jul 09 19:07:52.972866 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:52.972839    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f7885b79-hv86z_08d0246d-eac4-4424-b007-6ee9d2a34f1c/manager/0.log"
Jul 09 19:07:52.997386 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:52.997362    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f7885b79-hv86z_08d0246d-eac4-4424-b007-6ee9d2a34f1c/copy-manifests/0.log"
Jul 09 19:07:54.411552 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:54.411524    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rn4db_7a966206-99c8-4a39-968f-e92a3d860cdd/discovery/0.log"
Jul 09 19:07:54.463416 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:54.463383    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7dqvt_03aabde7-0427-4b32-afe1-dca90a34ae24/istio-proxy/0.log"
Jul 09 19:07:54.467306 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:54.467283    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-576fd7db4f-mx7hr_0a10a622-b903-4d9a-9167-7c96e310e88b/manager/0.log"
Jul 09 19:07:54.527965 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:54.527937    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7dqvt_03aabde7-0427-4b32-afe1-dca90a34ae24/istio-proxy/1.log"
Jul 09 19:07:54.588083 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:54.588051    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-569l6_8032829c-1601-4456-beea-406d065a820e/payload-processing/0.log"
Jul 09 19:07:56.063044 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:56.063006    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rn4db_7a966206-99c8-4a39-968f-e92a3d860cdd/discovery/0.log"
Jul 09 19:07:56.122026 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:56.121978    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7dqvt_03aabde7-0427-4b32-afe1-dca90a34ae24/istio-proxy/0.log"
Jul 09 19:07:56.180178 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:56.180152    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7dqvt_03aabde7-0427-4b32-afe1-dca90a34ae24/istio-proxy/1.log"
Jul 09 19:07:56.244292 ip-10-0-135-28 kubenswrapper[2566]: I0709 19:07:56.244264    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-569l6_8032829c-1601-4456-beea-406d065a820e/payload-processing/0.log"