Jul 10 02:39:50.076047 ip-10-0-143-93 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.452718    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456623    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456634    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456638    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456641    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456644    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456647    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456651    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:39:50.479222 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456654    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456658    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456661    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456664    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456667    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456670    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456673    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456675    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456678    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456681    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456684    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456687    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456689    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456692    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456695    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456698    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456701    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456705    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456707    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456710    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:39:50.480074 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456712    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456715    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456718    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456720    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456723    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456725    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456728    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456731    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456733    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456736    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456738    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456741    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456761    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456764    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456766    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456769    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456771    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456774    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456777    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:39:50.481160 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456781    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456786    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456789    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456792    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456795    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456797    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456800    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456804    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456808    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456811    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456814    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456817    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456820    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456823    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456826    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456829    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456832    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456835    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456838    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:39:50.482020 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456840    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456843    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456845    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456848    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456851    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456853    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456857    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456860    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456862    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456865    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456868    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456870    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456873    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456877    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456881    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456884    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456887    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456889    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456891    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:39:50.483349 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456894    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456897    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456899    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456902    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456905    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456907    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456910    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456912    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456915    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456918    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456920    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456924    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456927    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456929    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.456932    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457664    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457671    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457675    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457678    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457681    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:39:50.484013 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457684    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457687    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457690    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457693    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457695    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457698    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457700    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457704    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457706    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457709    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457711    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457713    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457716    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457718    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457721    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457724    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457726    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457729    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457733    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:39:50.485757 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457737    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457740    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457762    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457766    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457769    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457772    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457775    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457777    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457780    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457783    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457785    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457788    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457791    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457793    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457797    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457800    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457803    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457805    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457808    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:39:50.486405 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457810    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457814    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457818    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457821    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457824    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457827    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457830    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457834    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457837    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457842    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457845    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457848    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457851    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457853    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457856    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457859    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457861    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457871    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457874    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:39:50.487503 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457876    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457879    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457881    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457884    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457886    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457889    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457893    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457896    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457898    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457902    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457904    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457907    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457910    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457912    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457915    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457918    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457920    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457923    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457926    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457928    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:39:50.488153 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457930    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457934    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457938    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457942    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457945    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457948    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457950    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457953    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457955    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457958    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457960    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457963    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457966    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457968    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457971    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457973    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.457976    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458431    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458435    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458439    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:39:50.489060 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458441    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458444    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458447    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458449    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458452    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458455    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458457    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458460    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458463    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458465    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458468    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458471    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458473    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458476    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458480    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458482    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458485    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458487    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458490    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458493    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:39:50.490149 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458496    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458498    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458501    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458503    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458506    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458508    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458511    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458513    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458516    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458519    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458522    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458524    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458527    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458529    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458532    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458535    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458538    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458542    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458546    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458549    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:39:50.492315 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458552    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458554    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458557    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458560    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458563    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458565    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458569    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458572    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458574    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458577    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458579    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458582    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458586    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458589    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458591    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458596    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458599    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458602    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458605    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458608    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:39:50.493391 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458611    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458613    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458616    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458619    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458622    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458624    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458627    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458630    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458632    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458635    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458637    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458640    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458642    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458644    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458647    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458656    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458659    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458662    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458666    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:39:50.494520 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458669    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458671    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458674    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458677    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458680    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458683    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458685    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458688    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458691    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458693    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458696    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458698    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458701    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458703    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458706    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458708    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.458711    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458826    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458835    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458844    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458849    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 02:39:50.495181 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458855    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458858    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458863    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458868    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458871    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458874    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458877    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458883    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458886    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458889    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458893    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458897    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458900    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458903    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458908    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458910    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458914    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458917    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458920    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458924    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458928    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458931    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458934    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458937    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 02:39:50.504214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458941    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458944    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458946    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458949    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458953    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458956    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458959    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458962    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458965    2576 flags.go:64] FLAG: --enable-server="true"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458969    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458973    2576 flags.go:64] FLAG: --event-burst="100"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458976    2576 flags.go:64] FLAG: --event-qps="50"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458979    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458982    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.458998    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459004    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459008    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459011    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459017    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459020    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459023    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459026    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459029    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459032    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459035    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 02:39:50.505112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459037    2576 flags.go:64] FLAG: --feature-gates=""
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459041    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459045    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459048    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459051    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459054    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459058    2576 flags.go:64] FLAG: --help="false"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459061    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-143-93.ec2.internal"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459064    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459067    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459070    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459073    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459077    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459079    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459082    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459085    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459088    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459092    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459095    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459098    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459101    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459104    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459110    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459112    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 02:39:50.506838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459115    2576 flags.go:64] FLAG: --lock-file=""
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459120    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459123    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459126    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459132    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459135    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459138    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459141    2576 flags.go:64] FLAG: --logging-format="text"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459144    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459147    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459150    2576 flags.go:64] FLAG: --manifest-url=""
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459153    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459157    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459160    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459165    2576 flags.go:64] FLAG: --max-pods="110"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459168    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459171    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459174    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459176    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459180    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459182    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459185    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459193    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459196    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459199    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 02:39:50.507700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459203    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459206    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459212    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459215    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459219    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459223    2576 flags.go:64] FLAG: --port="10250"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459226    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459229    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ed6a4b9a9b2b2869"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459233    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459237    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459240    2576 flags.go:64] FLAG: --register-node="true"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459243    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459246    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459249    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459252    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459255    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459259    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459262    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459265    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459268    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459271    2576 flags.go:64] FLAG: --runonce="false"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459274    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459277    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459280    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459282    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 02:39:50.562502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459285    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 02:39:50.538294 ip-10-0-143-93 systemd[1]: Started Kubernetes Kubelet.
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459288    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459291    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459294    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459297    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459300    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459304    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459306    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459309    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459313    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459316    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459322    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459326    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459329    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459333    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459336    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459340    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459343    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459346    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459349    2576 flags.go:64] FLAG: --v="2"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459353    2576 flags.go:64] FLAG: --version="false"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459357    2576 flags.go:64] FLAG: --vmodule=""
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459361    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.459364    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459490    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:39:50.564449 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459494    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459497    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459502    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459506    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459509    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459512    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459515    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459517    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459520    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459522    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459525    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459527    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459530    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459533    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459535    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459538    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459541    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459544    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459547    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:39:50.567180 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459551    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459554    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459556    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459559    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459564    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459567    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459570    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459573    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459575    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459578    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459580    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459583    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459585    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459588    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459591    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459593    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459596    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459598    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459601    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459603    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:39:50.568220 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459606    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459608    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459611    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459613    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459616    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459619    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459621    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459624    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459626    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459629    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459632    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459635    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459638    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459641    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459644    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459646    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459650    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459654    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459658    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459661    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:39:50.570579 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459664    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459667    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459670    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459673    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459677    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459680    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459683    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459685    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459688    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459690    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459693    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459695    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459698    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459700    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459703    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459705    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459708    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459710    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459713    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459715    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:39:50.571236 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459718    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459720    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459723    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459726    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459730    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459732    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459735    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459737    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459741    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459761    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459764    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459767    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459770    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459773    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459775    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459778    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459780    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459783    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:39:50.573350 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.459785    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.460286    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.468379    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.468401    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.469470    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.469545    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.470588    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.470658    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.470819    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.470824    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.470840    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.470882    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.470963    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471048    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471051    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:39:50.573974 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471054    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471056    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471083    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471269    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471273    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471275    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471278    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471280    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471283    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471351    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471430    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471502    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471505    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471508    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471511    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471537    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471721    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471724    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471729    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471940    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:39:50.577163 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471945    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.471947    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472096    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472157    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472160    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472163    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472166    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472370    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472374    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472376    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472379    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472394    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472572    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472599    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472603    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472678    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472823    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472826    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472829    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:39:50.578238 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472834    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472837    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472840    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472842    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472847    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472850    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472853    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472858    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472861    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472864    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472868    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472873    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472875    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472880    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472883    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472886    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472889    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472895    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472898    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:39:50.581159 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472901    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472906    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472909    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472912    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472916    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472919    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472922    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472925    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472929    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472933    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472935    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472940    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472943    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472945    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472948    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472954    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472958    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472963    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472966    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472968    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:39:50.582041 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472971    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472975    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472978    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472980    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472987    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472990    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472993    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.472995    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473000    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473002    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473005    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473009    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.473014    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473176    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473181    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:39:50.584626 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473186    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473189    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473192    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473197    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473199    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473202    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473204    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473209    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473212    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473215    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473219    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473222    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473224    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473227    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473231    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473235    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473241    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473244    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473247    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:39:50.585451 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473251    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473254    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473257    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473262    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473265    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473268    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473271    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473276    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473278    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473281    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473283    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473290    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473294    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473299    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473303    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473306    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473310    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473314    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473316    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:39:50.587554 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473319    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473324    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473326    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473329    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473334    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473337    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473339    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473342    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473346    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473349    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473351    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473354    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473358    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473361    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473363    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473366    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473372    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473374    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473377    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473381    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:39:50.588167 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473383    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473386    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473389    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473393    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473396    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473400    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473405    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473407    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473410    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473413    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473417    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473420    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473422    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473426    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473429    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473431    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473434    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473437    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473439    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:39:50.589067 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473442    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473444    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473448    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473451    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473453    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473456    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473460    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473463    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473466    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473470    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473475    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473477    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473480    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473485    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473488    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473490    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473493    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473497    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473503    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:39:50.589680 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:50.473505    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.473513    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.473667    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.476927    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.478544    2576 server.go:1019] "Starting client certificate rotation"
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.478652    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.478687    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.480979    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.491533    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.501979    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.504727    2576 log.go:25] "Validated CRI v1 image API"
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.506234    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 02:39:50.678432 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.508881    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 02:39:50.679418 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.509378    2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 93da09dc-b7ac-4293-9d0d-323fa6369984:/dev/nvme0n1p4 aeaf3df0-9d5f-4b9b-9665-ca500db30457:/dev/nvme0n1p3]
Jul 10 02:39:50.679418 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.509397    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.514726    2576 manager.go:216] Machine: {Timestamp:2026-07-10 02:39:50.513451317 +0000 UTC m=+0.322716611 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102724 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2516dc08a864c64ea69eac184288ae SystemUUID:ec2516dc-08a8-64c6-4ea6-9eac184288ae BootID:49698baf-4972-46fb-bbbd-c0fcbd2ca6f2 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:db:79:d9:60:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:db:79:d9:60:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:8d:65:65:dc:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.514865    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.514957    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.518792    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.518837    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-93.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.519705    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.519719    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.519730    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.520574    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.522222    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.525687    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.525879    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.525944    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.525974    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.525993    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.526007    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.526020    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 02:39:50.681476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.526497    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.526517    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.528943    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.531410    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.531451    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.531632    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.531647    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.531655    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.531662    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.531675    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.531683    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.532111    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.532119    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.532126    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.532132    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.532146    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.532153    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.533778    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.533793    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.537361    2576 server.go:1268] "Started kubelet"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.537453    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.537689    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.537758    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.542281    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.542363    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.542486    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.542613    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.544780    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.545925    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.548755    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-93.ec2.internal.18c0cd4ebf2e03a4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-93.ec2.internal,UID:ip-10-0-143-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-93.ec2.internal,},FirstTimestamp:2026-07-10 02:39:50.537323428 +0000 UTC m=+0.346588721,LastTimestamp:2026-07-10 02:39:50.537323428 +0000 UTC m=+0.346588721,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-93.ec2.internal,}"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.553309    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.553309    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.553419    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554007    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554028    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554039    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554146    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554172    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554453    2576 factory.go:55] Registering systemd factory
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554468    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554710    2576 factory.go:153] Registering CRI-O factory
Jul 10 02:39:50.683644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554726    2576 factory.go:223] Registration of the crio container factory successfully
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554797    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554793    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fffr9"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554822    2576 factory.go:103] Registering Raw factory
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.554855    2576 manager.go:1196] Started watching for new ooms in manager
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.555310    2576 manager.go:349] Starting recovery of all containers
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.563124    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.563370    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.563376    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.564594    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fffr9"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.566407    2576 manager.go:354] Recovery completed
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.571778    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.574352    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.574377    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.574389    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.575003    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.574996    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.575089    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.575107    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.575863    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.575896    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.575909    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.576314    2576 policy_none.go:49] "None policy: Start"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.576327    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.576338    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.577300    2576 policy_none.go:47] "Start"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.643605    2576 manager.go:337] "Starting Device Plugin manager"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.643702    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.643714    2576 server.go:87] "Starting device plugin registration server"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.644049    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.644061    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.644161    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.644286    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.644299    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.644939    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 02:39:50.684846 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.644986    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:50.775018 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.772707    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:39:50.775018 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.774238    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:39:50.775018 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.774270    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:39:50.775018 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.774290    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:39:50.775018 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.774328    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-93.ec2.internal"
Jul 10 02:39:50.860383 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.778574    2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-93.ec2.internal\" not found" node="ip-10-0-143-93.ec2.internal"
Jul 10 02:39:50.860383 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.785559    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-93.ec2.internal"
Jul 10 02:39:50.860383 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.785580    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-93.ec2.internal\": node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:50.860383 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.792285    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 02:39:50.860383 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.792316    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 02:39:50.860383 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.792335    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 02:39:50.860383 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.792365    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 02:39:50.860383 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.794726    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:39:50.860383 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.811794    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:50.893539 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.893496    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal"]
Jul 10 02:39:50.893644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.893592    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:39:50.894914 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.894898    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:39:50.894968 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.894931    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:39:50.894968 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.894942    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:39:50.896320 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.896307    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:39:50.896440 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.896418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:50.896502 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.896463    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:39:50.897258 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.897237    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:39:50.897377 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.897274    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:39:50.897377 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.897277    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:39:50.897377 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.897289    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:39:50.897377 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.897300    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:39:50.897377 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.897312    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:39:50.898530 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.898513    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:50.898602 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.898539    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:39:50.899287 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.899272    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:39:50.899354 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.899308    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:39:50.899354 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.899324    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:39:50.912174 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.912150    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:50.926729 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.926706    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-93.ec2.internal\" not found" node="ip-10-0-143-93.ec2.internal"
Jul 10 02:39:50.931617 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:50.931602    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-93.ec2.internal\" not found" node="ip-10-0-143-93.ec2.internal"
Jul 10 02:39:50.989431 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.989402    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75342c7e3311385e2be3c26bd0e39bb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal\" (UID: \"75342c7e3311385e2be3c26bd0e39bb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:50.989548 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.989435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7267bfa133f7ae1ca9f615a6770bb5df-config\") pod \"kube-apiserver-proxy-ip-10-0-143-93.ec2.internal\" (UID: \"7267bfa133f7ae1ca9f615a6770bb5df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:50.989548 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:50.989455    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/75342c7e3311385e2be3c26bd0e39bb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal\" (UID: \"75342c7e3311385e2be3c26bd0e39bb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.012940 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:51.012869    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:51.092373 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.092335    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/75342c7e3311385e2be3c26bd0e39bb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal\" (UID: \"75342c7e3311385e2be3c26bd0e39bb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.092373 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.092377    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75342c7e3311385e2be3c26bd0e39bb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal\" (UID: \"75342c7e3311385e2be3c26bd0e39bb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.092487 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.092403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7267bfa133f7ae1ca9f615a6770bb5df-config\") pod \"kube-apiserver-proxy-ip-10-0-143-93.ec2.internal\" (UID: \"7267bfa133f7ae1ca9f615a6770bb5df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.092487 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.092454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7267bfa133f7ae1ca9f615a6770bb5df-config\") pod \"kube-apiserver-proxy-ip-10-0-143-93.ec2.internal\" (UID: \"7267bfa133f7ae1ca9f615a6770bb5df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.092487 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.092470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75342c7e3311385e2be3c26bd0e39bb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal\" (UID: \"75342c7e3311385e2be3c26bd0e39bb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.092597 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.092454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/75342c7e3311385e2be3c26bd0e39bb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal\" (UID: \"75342c7e3311385e2be3c26bd0e39bb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.113434 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:51.113396    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:51.214760 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:51.214708    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:51.230029 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.230001    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.235606 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.235578    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.315515 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:51.315398    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:51.416083 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:51.416052    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:51.478320 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.478289    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 02:39:51.478485 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.478463    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 02:39:51.516941 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:51.516913    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:51.568905 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.568812    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 02:34:51 +0000 UTC" deadline="2028-03-10 10:30:19.408921143 +0000 UTC"
Jul 10 02:39:51.568905 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.568842    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14623h50m27.840082213s"
Jul 10 02:39:51.617697 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:51.617657    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-93.ec2.internal\" not found"
Jul 10 02:39:51.705212 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.705187    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:39:51.714081 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.714057    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:39:51.754519 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.754481    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.768098 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.768070    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 02:39:51.770138 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.770119    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal"
Jul 10 02:39:51.782298 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.782267    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 02:39:51.801068 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:51.801020    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75342c7e3311385e2be3c26bd0e39bb2.slice/crio-64b10be48f1c258b48d47340365e9378ea1bef4ecc22609becbe47b471466404 WatchSource:0}: Error finding container 64b10be48f1c258b48d47340365e9378ea1bef4ecc22609becbe47b471466404: Status 404 returned error can't find the container with id 64b10be48f1c258b48d47340365e9378ea1bef4ecc22609becbe47b471466404
Jul 10 02:39:51.801534 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:51.801513    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7267bfa133f7ae1ca9f615a6770bb5df.slice/crio-fc755611c77da61e383f9cb4e2f87cbaa2fb8c37609d609a8f32faa7a0fad5be WatchSource:0}: Error finding container fc755611c77da61e383f9cb4e2f87cbaa2fb8c37609d609a8f32faa7a0fad5be: Status 404 returned error can't find the container with id fc755611c77da61e383f9cb4e2f87cbaa2fb8c37609d609a8f32faa7a0fad5be
Jul 10 02:39:51.807101 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:51.807086    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:39:52.019252 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.019162    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:39:52.526911 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.526875    2576 apiserver.go:52] "Watching apiserver"
Jul 10 02:39:52.535597 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.535568    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 02:39:52.536152 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.536111    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-qqvdz","openshift-multus/multus-additional-cni-plugins-cmb92","openshift-multus/multus-k2s62","openshift-network-diagnostics/network-check-target-q4q57","kube-system/global-pull-secret-syncer-npnft","openshift-dns/node-resolver-cfmzm","openshift-image-registry/node-ca-r7k5d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal","openshift-multus/network-metrics-daemon-57zjp","openshift-network-operator/iptables-alerter-872cm","openshift-ovn-kubernetes/ovnkube-node-ln8cg","kube-system/konnectivity-agent-gphcf","kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"]
Jul 10 02:39:52.537806 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.537773    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.541681 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.541654    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.542166 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.542133    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j7qrb\""
Jul 10 02:39:52.542279 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.542166    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 02:39:52.542545 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.542523    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 02:39:52.542665 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.542643    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.542728 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.542711    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 02:39:52.544074 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.544047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:39:52.544187 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.544121    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:39:52.545574 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.545342    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 02:39:52.545574 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.545413    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 02:39:52.545574 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.545413    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 02:39:52.545574 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.545349    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q46k4\""
Jul 10 02:39:52.545876 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.545709    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 02:39:52.545876 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.545787    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 02:39:52.545876 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.545816    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 02:39:52.546008 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.545991    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 02:39:52.546063 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.546048    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:52.546122 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.546092    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cxs44\""
Jul 10 02:39:52.546173 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.546137    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:39:52.546228 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.546208    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 02:39:52.546282 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.546237    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 02:39:52.546335 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.546283    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 02:39:52.547524 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.547265    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.548743 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.548596    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:52.548743 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.548664    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:39:52.549537 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.549516    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 02:39:52.549625 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.549562    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6mgjg\""
Jul 10 02:39:52.550735 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.550046    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.551137 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.551082    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 02:39:52.551469 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.551443    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.552790 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.552756    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 02:39:52.553041 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.553018    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 02:39:52.553147 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.553075    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 02:39:52.553267 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.553248    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5vwhx\""
Jul 10 02:39:52.553446 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.553429    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.553608 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.553590    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 02:39:52.554981 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.554005    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pnvpt\""
Jul 10 02:39:52.554981 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.554636    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 02:39:52.554981 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.554898    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 02:39:52.554981 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.554964    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.556864 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.556836    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z256g\""
Jul 10 02:39:52.557019 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.556888    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:39:52.557019 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.556903    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 02:39:52.557137 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.557080    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:39:52.557197 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.556908    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 02:39:52.557498 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.557439    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:39:52.557666 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.557630    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-64knq\""
Jul 10 02:39:52.557666 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.557648    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 02:39:52.559379 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.559360    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rgp28\""
Jul 10 02:39:52.559700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.559586    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 02:39:52.559912 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.559889    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 02:39:52.569954 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.569927    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 02:39:52.600886 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.600859    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8hfzt"
Jul 10 02:39:52.607704 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.607676    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8hfzt"
Jul 10 02:39:52.614908 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.614886    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-slash\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615036 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.614919    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-var-lib-openvswitch\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615036 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.614943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-run-ovn\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615036 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.614958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-node-log\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615036 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.614973    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-cni-bin\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615036 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615004    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-os-release\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.615036 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-system-cni-dir\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.615355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615039    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-var-lib-kubelet\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.615355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-systemd-units\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615104    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/228cc28c-9b29-4a85-a465-040c721653fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.615355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-multus-cni-dir\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.615355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615153    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-run-k8s-cni-cncf-io\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.615355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615177    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-var-lib-cni-bin\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.615355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/09ab858c-346a-4ccd-9834-627796c08d79-multus-daemon-config\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.615355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:52.615355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmpzn\" (UniqueName: \"kubernetes.io/projected/01980251-990d-4898-b259-8a05b74de2fc-kube-api-access-hmpzn\") pod \"node-ca-r7k5d\" (UID: \"01980251-990d-4898-b259-8a05b74de2fc\") " pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.615355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615340    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-run-openvswitch\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615368    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-run-ovn-kubernetes\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615391    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/61046ad7-86b4-4781-9728-e7710e740f72-ovnkube-config\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/61046ad7-86b4-4781-9728-e7710e740f72-env-overrides\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-multus-socket-dir-parent\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615465    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-run-multus-certs\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615510    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/61046ad7-86b4-4781-9728-e7710e740f72-ovnkube-script-lib\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615536    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-sys\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-tuned\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615589    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-cnibin\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w77zn\" (UniqueName: \"kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn\") pod \"network-check-target-q4q57\" (UID: \"89160553-786f-443a-8e3b-d6ffb4f4b761\") " pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615664    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/22008c3f-7d3c-4151-910c-8950c916a84c-hosts-file\") pod \"node-resolver-cfmzm\" (UID: \"22008c3f-7d3c-4151-910c-8950c916a84c\") " pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615713    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22008c3f-7d3c-4151-910c-8950c916a84c-tmp-dir\") pod \"node-resolver-cfmzm\" (UID: \"22008c3f-7d3c-4151-910c-8950c916a84c\") " pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615741    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-etc-openvswitch\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-system-cni-dir\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.615979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/528c8356-c22a-40a3-a768-7a7c90c6c44f-iptables-alerter-script\") pod \"iptables-alerter-872cm\" (UID: \"528c8356-c22a-40a3-a768-7a7c90c6c44f\") " pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwzrt\" (UniqueName: \"kubernetes.io/projected/528c8356-c22a-40a3-a768-7a7c90c6c44f-kube-api-access-gwzrt\") pod \"iptables-alerter-872cm\" (UID: \"528c8356-c22a-40a3-a768-7a7c90c6c44f\") " pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615872    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-var-lib-cni-multus\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615942    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzz4q\" (UniqueName: \"kubernetes.io/projected/09ab858c-346a-4ccd-9834-627796c08d79-kube-api-access-qzz4q\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.615976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqnrm\" (UniqueName: \"kubernetes.io/projected/22008c3f-7d3c-4151-910c-8950c916a84c-kube-api-access-zqnrm\") pod \"node-resolver-cfmzm\" (UID: \"22008c3f-7d3c-4151-910c-8950c916a84c\") " pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616012    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-kubelet\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg5gp\" (UniqueName: \"kubernetes.io/projected/61046ad7-86b4-4781-9728-e7710e740f72-kube-api-access-xg5gp\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616098    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-registration-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616126    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-modprobe-d\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616153    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-kubernetes\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-sysctl-d\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-host\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616233    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09ab858c-346a-4ccd-9834-627796c08d79-cni-binary-copy\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-etc-kubernetes\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616267    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-run-systemd\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.616765 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616286    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-cni-netd\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-device-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616348    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-cnibin\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616381    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/528c8356-c22a-40a3-a768-7a7c90c6c44f-host-slash\") pod \"iptables-alerter-872cm\" (UID: \"528c8356-c22a-40a3-a768-7a7c90c6c44f\") " pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-run\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616425    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/228cc28c-9b29-4a85-a465-040c721653fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616442    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-sysconfig\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616464    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01980251-990d-4898-b259-8a05b74de2fc-host\") pod \"node-ca-r7k5d\" (UID: \"01980251-990d-4898-b259-8a05b74de2fc\") " pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/01980251-990d-4898-b259-8a05b74de2fc-serviceca\") pod \"node-ca-r7k5d\" (UID: \"01980251-990d-4898-b259-8a05b74de2fc\") " pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616518    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-sys-fs\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616555    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7zhn\" (UniqueName: \"kubernetes.io/projected/228cc28c-9b29-4a85-a465-040c721653fa-kube-api-access-p7zhn\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616576    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-sysctl-conf\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-systemd\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjntx\" (UniqueName: \"kubernetes.io/projected/cb9fe830-29c0-4665-beea-ffda1ca5ef53-kube-api-access-jjntx\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616649    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/61046ad7-86b4-4781-9728-e7710e740f72-ovn-node-metrics-cert\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.617594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616668    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-socket-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616688    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpkjc\" (UniqueName: \"kubernetes.io/projected/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-kube-api-access-hpkjc\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616712    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-multus-conf-dir\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d813814-8b23-41d4-852b-ff842e8a5771-kubelet-config\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616762    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d813814-8b23-41d4-852b-ff842e8a5771-dbus\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-run-netns\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616823    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-etc-selinux\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-var-lib-kubelet\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-os-release\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-run-netns\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616936    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-hostroot\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-log-socket\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.616985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86jlx\" (UniqueName: \"kubernetes.io/projected/f6649dea-7a4b-435f-9696-ef409686f070-kube-api-access-86jlx\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.617009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/228cc28c-9b29-4a85-a465-040c721653fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.617033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.617056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-tmp\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.618523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.617084    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-lib-modules\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.655073 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.655044    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 02:39:52.691764 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.691709    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:39:52.717325 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.717514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-sys-fs\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.717514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7zhn\" (UniqueName: \"kubernetes.io/projected/228cc28c-9b29-4a85-a465-040c721653fa-kube-api-access-p7zhn\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.717514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-sysctl-conf\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.717514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717416    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-systemd\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.717514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjntx\" (UniqueName: \"kubernetes.io/projected/cb9fe830-29c0-4665-beea-ffda1ca5ef53-kube-api-access-jjntx\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:52.717514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717468    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/61046ad7-86b4-4781-9728-e7710e740f72-ovn-node-metrics-cert\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.717514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-socket-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpkjc\" (UniqueName: \"kubernetes.io/projected/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-kube-api-access-hpkjc\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-multus-conf-dir\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717573    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d813814-8b23-41d4-852b-ff842e8a5771-kubelet-config\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d813814-8b23-41d4-852b-ff842e8a5771-dbus\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717667    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10-konnectivity-ca\") pod \"konnectivity-agent-gphcf\" (UID: \"a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10\") " pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717693    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-run-netns\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717724    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-etc-selinux\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-var-lib-kubelet\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-os-release\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717826    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-run-netns\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-hostroot\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-log-socket\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86jlx\" (UniqueName: \"kubernetes.io/projected/f6649dea-7a4b-435f-9696-ef409686f070-kube-api-access-86jlx\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/228cc28c-9b29-4a85-a465-040c721653fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717952    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-sys-fs\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.718048    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-var-lib-kubelet\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.718182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.718138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-os-release\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.719040 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.718165    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.719040 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.718184    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-run-netns\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.719040 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.718229    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-hostroot\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.719040 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.718270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-log-socket\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.719040 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.718292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-etc-selinux\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.719040 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.718308    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.719040 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.718464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-sysctl-conf\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.719040 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.718750    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-systemd\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.719415 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.719237    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/228cc28c-9b29-4a85-a465-040c721653fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.719415 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.719304    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-run-netns\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.719415 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.719318    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 02:39:52.719415 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.719364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d813814-8b23-41d4-852b-ff842e8a5771-kubelet-config\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:52.719603 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.719455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-socket-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.719703 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.719681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-multus-conf-dir\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.719941 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.719922    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d813814-8b23-41d4-852b-ff842e8a5771-dbus\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:52.720013 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.717960    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.720013 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.719976    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-tmp\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.720013 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-lib-modules\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.720180 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-slash\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720180 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-var-lib-openvswitch\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720180 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720082    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-run-ovn\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720180 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-node-log\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720180 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-cni-bin\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-os-release\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.720434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-system-cni-dir\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720229    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-var-lib-kubelet\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-systemd-units\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720283    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/228cc28c-9b29-4a85-a465-040c721653fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.720434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720310    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-multus-cni-dir\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-run-k8s-cni-cncf-io\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720367    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-var-lib-cni-bin\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/09ab858c-346a-4ccd-9834-627796c08d79-multus-daemon-config\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmpzn\" (UniqueName: \"kubernetes.io/projected/01980251-990d-4898-b259-8a05b74de2fc-kube-api-access-hmpzn\") pod \"node-ca-r7k5d\" (UID: \"01980251-990d-4898-b259-8a05b74de2fc\") " pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-run-openvswitch\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-run-ovn-kubernetes\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/61046ad7-86b4-4781-9728-e7710e740f72-ovnkube-config\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720557    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/61046ad7-86b4-4781-9728-e7710e740f72-env-overrides\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-multus-socket-dir-parent\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-run-multus-certs\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/61046ad7-86b4-4781-9728-e7710e740f72-ovnkube-script-lib\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-sys\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720752    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-sys\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-multus-cni-dir\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-run-ovn-kubernetes\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-run-k8s-cni-cncf-io\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-tuned\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720857    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-multus-socket-dir-parent\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720858    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-var-lib-cni-bin\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.720947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.720882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-cnibin\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.720986    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.721072    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs podName:cb9fe830-29c0-4665-beea-ffda1ca5ef53 nodeName:}" failed. No retries permitted until 2026-07-10 02:39:53.221048662 +0000 UTC m=+3.030313968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs") pod "network-metrics-daemon-57zjp" (UID: "cb9fe830-29c0-4665-beea-ffda1ca5ef53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.720989    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721146    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-cnibin\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.721173    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret podName:3d813814-8b23-41d4-852b-ff842e8a5771 nodeName:}" failed. No retries permitted until 2026-07-10 02:39:53.221153583 +0000 UTC m=+3.030418867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret") pod "global-pull-secret-syncer-npnft" (UID: "3d813814-8b23-41d4-852b-ff842e8a5771") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-run-multus-certs\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721201    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77zn\" (UniqueName: \"kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn\") pod \"network-check-target-q4q57\" (UID: \"89160553-786f-443a-8e3b-d6ffb4f4b761\") " pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/22008c3f-7d3c-4151-910c-8950c916a84c-hosts-file\") pod \"node-resolver-cfmzm\" (UID: \"22008c3f-7d3c-4151-910c-8950c916a84c\") " pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22008c3f-7d3c-4151-910c-8950c916a84c-tmp-dir\") pod \"node-resolver-cfmzm\" (UID: \"22008c3f-7d3c-4151-910c-8950c916a84c\") " pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-etc-openvswitch\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-system-cni-dir\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/528c8356-c22a-40a3-a768-7a7c90c6c44f-iptables-alerter-script\") pod \"iptables-alerter-872cm\" (UID: \"528c8356-c22a-40a3-a768-7a7c90c6c44f\") " pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwzrt\" (UniqueName: \"kubernetes.io/projected/528c8356-c22a-40a3-a768-7a7c90c6c44f-kube-api-access-gwzrt\") pod \"iptables-alerter-872cm\" (UID: \"528c8356-c22a-40a3-a768-7a7c90c6c44f\") " pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-var-lib-cni-multus\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721403    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/09ab858c-346a-4ccd-9834-627796c08d79-multus-daemon-config\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721415    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-cni-bin\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.721833 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721429    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzz4q\" (UniqueName: \"kubernetes.io/projected/09ab858c-346a-4ccd-9834-627796c08d79-kube-api-access-qzz4q\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721457    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqnrm\" (UniqueName: \"kubernetes.io/projected/22008c3f-7d3c-4151-910c-8950c916a84c-kube-api-access-zqnrm\") pod \"node-resolver-cfmzm\" (UID: \"22008c3f-7d3c-4151-910c-8950c916a84c\") " pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721490    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10-agent-certs\") pod \"konnectivity-agent-gphcf\" (UID: \"a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10\") " pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721491    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-etc-openvswitch\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-kubelet\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721555    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/61046ad7-86b4-4781-9728-e7710e740f72-ovnkube-script-lib\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721566    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg5gp\" (UniqueName: \"kubernetes.io/projected/61046ad7-86b4-4781-9728-e7710e740f72-kube-api-access-xg5gp\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-registration-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-modprobe-d\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-kubernetes\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721734    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-system-cni-dir\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721744    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-sysctl-d\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-host\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09ab858c-346a-4ccd-9834-627796c08d79-cni-binary-copy\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721848    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-etc-kubernetes\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721886    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-run-systemd\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.722751 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721917    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-cni-netd\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-device-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.721971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-cnibin\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/528c8356-c22a-40a3-a768-7a7c90c6c44f-host-slash\") pod \"iptables-alerter-872cm\" (UID: \"528c8356-c22a-40a3-a768-7a7c90c6c44f\") " pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722025    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-run\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-modprobe-d\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/228cc28c-9b29-4a85-a465-040c721653fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722078    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/61046ad7-86b4-4781-9728-e7710e740f72-ovnkube-config\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722084    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-sysconfig\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722097    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/22008c3f-7d3c-4151-910c-8950c916a84c-hosts-file\") pod \"node-resolver-cfmzm\" (UID: \"22008c3f-7d3c-4151-910c-8950c916a84c\") " pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722112    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01980251-990d-4898-b259-8a05b74de2fc-host\") pod \"node-ca-r7k5d\" (UID: \"01980251-990d-4898-b259-8a05b74de2fc\") " pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722138    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/01980251-990d-4898-b259-8a05b74de2fc-serviceca\") pod \"node-ca-r7k5d\" (UID: \"01980251-990d-4898-b259-8a05b74de2fc\") " pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722143    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-run-openvswitch\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-etc-kubernetes\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722249    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-device-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-cnibin\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/528c8356-c22a-40a3-a768-7a7c90c6c44f-iptables-alerter-script\") pod \"iptables-alerter-872cm\" (UID: \"528c8356-c22a-40a3-a768-7a7c90c6c44f\") " pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722330    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/528c8356-c22a-40a3-a768-7a7c90c6c44f-host-slash\") pod \"iptables-alerter-872cm\" (UID: \"528c8356-c22a-40a3-a768-7a7c90c6c44f\") " pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.723624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722383    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-run\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22008c3f-7d3c-4151-910c-8950c916a84c-tmp-dir\") pod \"node-resolver-cfmzm\" (UID: \"22008c3f-7d3c-4151-910c-8950c916a84c\") " pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-kubelet\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722593    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-var-lib-cni-multus\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722775    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-sysconfig\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722816    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09ab858c-346a-4ccd-9834-627796c08d79-cni-binary-copy\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722916    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-run-systemd\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722952    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01980251-990d-4898-b259-8a05b74de2fc-host\") pod \"node-ca-r7k5d\" (UID: \"01980251-990d-4898-b259-8a05b74de2fc\") " pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722964    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-lib-modules\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.722970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-kubernetes\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-cni-netd\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-systemd-units\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723056    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-host-var-lib-kubelet\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723092    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-host-slash\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/228cc28c-9b29-4a85-a465-040c721653fa-os-release\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-var-lib-openvswitch\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-run-ovn\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/61046ad7-86b4-4781-9728-e7710e740f72-node-log\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.724631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723262    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-registration-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.725516 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723319    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09ab858c-346a-4ccd-9834-627796c08d79-system-cni-dir\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.725516 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723373    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-host\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.725516 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723438    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-sysctl-d\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.725516 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723490    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6649dea-7a4b-435f-9696-ef409686f070-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.725516 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/228cc28c-9b29-4a85-a465-040c721653fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.725516 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.723848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/01980251-990d-4898-b259-8a05b74de2fc-serviceca\") pod \"node-ca-r7k5d\" (UID: \"01980251-990d-4898-b259-8a05b74de2fc\") " pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.725516 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.724108    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/228cc28c-9b29-4a85-a465-040c721653fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.725516 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.724547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/61046ad7-86b4-4781-9728-e7710e740f72-env-overrides\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.725516 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.725350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-tmp\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.725516 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.725475    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/61046ad7-86b4-4781-9728-e7710e740f72-ovn-node-metrics-cert\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.730190 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.729615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjntx\" (UniqueName: \"kubernetes.io/projected/cb9fe830-29c0-4665-beea-ffda1ca5ef53-kube-api-access-jjntx\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:52.730190 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.729743    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:39:52.730190 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.729787    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:39:52.730190 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.729804    2576 projected.go:196] Error preparing data for projected volume kube-api-access-w77zn for pod openshift-network-diagnostics/network-check-target-q4q57: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:39:52.730190 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:52.729883    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn podName:89160553-786f-443a-8e3b-d6ffb4f4b761 nodeName:}" failed. No retries permitted until 2026-07-10 02:39:53.229863414 +0000 UTC m=+3.039128722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w77zn" (UniqueName: "kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn") pod "network-check-target-q4q57" (UID: "89160553-786f-443a-8e3b-d6ffb4f4b761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:39:52.730190 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.729883    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-etc-tuned\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.730190 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.730144    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86jlx\" (UniqueName: \"kubernetes.io/projected/f6649dea-7a4b-435f-9696-ef409686f070-kube-api-access-86jlx\") pod \"aws-ebs-csi-driver-node-l5ql8\" (UID: \"f6649dea-7a4b-435f-9696-ef409686f070\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.732248 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.732154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmpzn\" (UniqueName: \"kubernetes.io/projected/01980251-990d-4898-b259-8a05b74de2fc-kube-api-access-hmpzn\") pod \"node-ca-r7k5d\" (UID: \"01980251-990d-4898-b259-8a05b74de2fc\") " pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.735635 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.735388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqnrm\" (UniqueName: \"kubernetes.io/projected/22008c3f-7d3c-4151-910c-8950c916a84c-kube-api-access-zqnrm\") pod \"node-resolver-cfmzm\" (UID: \"22008c3f-7d3c-4151-910c-8950c916a84c\") " pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.735635 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.735580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpkjc\" (UniqueName: \"kubernetes.io/projected/6adfb3eb-939c-4bd9-8a87-2e66b89b9ece-kube-api-access-hpkjc\") pod \"tuned-qqvdz\" (UID: \"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.736467 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.736440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7zhn\" (UniqueName: \"kubernetes.io/projected/228cc28c-9b29-4a85-a465-040c721653fa-kube-api-access-p7zhn\") pod \"multus-additional-cni-plugins-cmb92\" (UID: \"228cc28c-9b29-4a85-a465-040c721653fa\") " pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.737964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.737927    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwzrt\" (UniqueName: \"kubernetes.io/projected/528c8356-c22a-40a3-a768-7a7c90c6c44f-kube-api-access-gwzrt\") pod \"iptables-alerter-872cm\" (UID: \"528c8356-c22a-40a3-a768-7a7c90c6c44f\") " pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.738483 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.738448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg5gp\" (UniqueName: \"kubernetes.io/projected/61046ad7-86b4-4781-9728-e7710e740f72-kube-api-access-xg5gp\") pod \"ovnkube-node-ln8cg\" (UID: \"61046ad7-86b4-4781-9728-e7710e740f72\") " pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.739154 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.738846    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzz4q\" (UniqueName: \"kubernetes.io/projected/09ab858c-346a-4ccd-9834-627796c08d79-kube-api-access-qzz4q\") pod \"multus-k2s62\" (UID: \"09ab858c-346a-4ccd-9834-627796c08d79\") " pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.798024 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.797904    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal" event={"ID":"75342c7e3311385e2be3c26bd0e39bb2","Type":"ContainerStarted","Data":"64b10be48f1c258b48d47340365e9378ea1bef4ecc22609becbe47b471466404"}
Jul 10 02:39:52.799215 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.799185    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal" event={"ID":"7267bfa133f7ae1ca9f615a6770bb5df","Type":"ContainerStarted","Data":"fc755611c77da61e383f9cb4e2f87cbaa2fb8c37609d609a8f32faa7a0fad5be"}
Jul 10 02:39:52.822689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.822645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10-agent-certs\") pod \"konnectivity-agent-gphcf\" (UID: \"a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10\") " pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:39:52.822893 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.822718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10-konnectivity-ca\") pod \"konnectivity-agent-gphcf\" (UID: \"a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10\") " pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:39:52.823378 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.823349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10-konnectivity-ca\") pod \"konnectivity-agent-gphcf\" (UID: \"a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10\") " pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:39:52.825326 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.825300    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10-agent-certs\") pod \"konnectivity-agent-gphcf\" (UID: \"a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10\") " pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:39:52.855738 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.855692    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r7k5d"
Jul 10 02:39:52.864115 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.864082    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:39:52.869193 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:52.869161    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01980251_990d_4898_b259_8a05b74de2fc.slice/crio-de2d48d68ca768a24644f523affc142bfb4c18fa79fc145cc58c107537da7151 WatchSource:0}: Error finding container de2d48d68ca768a24644f523affc142bfb4c18fa79fc145cc58c107537da7151: Status 404 returned error can't find the container with id de2d48d68ca768a24644f523affc142bfb4c18fa79fc145cc58c107537da7151
Jul 10 02:39:52.874228 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:52.874204    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61046ad7_86b4_4781_9728_e7710e740f72.slice/crio-aa305232788c49b0d114a3cd3a946ebf3f4ccfdd2caedcc8f2ae72be2f3aaab9 WatchSource:0}: Error finding container aa305232788c49b0d114a3cd3a946ebf3f4ccfdd2caedcc8f2ae72be2f3aaab9: Status 404 returned error can't find the container with id aa305232788c49b0d114a3cd3a946ebf3f4ccfdd2caedcc8f2ae72be2f3aaab9
Jul 10 02:39:52.875155 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.875123    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k2s62"
Jul 10 02:39:52.880927 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.880893    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cfmzm"
Jul 10 02:39:52.891503 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.891393    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8"
Jul 10 02:39:52.891724 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:52.891694    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22008c3f_7d3c_4151_910c_8950c916a84c.slice/crio-4219d63050a79b303936da7e9d85917d9b9305d50b22d2208a88ba11402dd3eb WatchSource:0}: Error finding container 4219d63050a79b303936da7e9d85917d9b9305d50b22d2208a88ba11402dd3eb: Status 404 returned error can't find the container with id 4219d63050a79b303936da7e9d85917d9b9305d50b22d2208a88ba11402dd3eb
Jul 10 02:39:52.900239 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.900212    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cmb92"
Jul 10 02:39:52.901144 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:52.901115    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6649dea_7a4b_435f_9696_ef409686f070.slice/crio-dcc393a6926208d4dbd1fc69695ce6a2b265b8a0ce3d9ce5e2f7037f879b9ef4 WatchSource:0}: Error finding container dcc393a6926208d4dbd1fc69695ce6a2b265b8a0ce3d9ce5e2f7037f879b9ef4: Status 404 returned error can't find the container with id dcc393a6926208d4dbd1fc69695ce6a2b265b8a0ce3d9ce5e2f7037f879b9ef4
Jul 10 02:39:52.909235 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:52.909209    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod228cc28c_9b29_4a85_a465_040c721653fa.slice/crio-2b78fd37230d6487398d25233fffc06cb000a8598890e643f8c3358a672c70b8 WatchSource:0}: Error finding container 2b78fd37230d6487398d25233fffc06cb000a8598890e643f8c3358a672c70b8: Status 404 returned error can't find the container with id 2b78fd37230d6487398d25233fffc06cb000a8598890e643f8c3358a672c70b8
Jul 10 02:39:52.910195 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.910173    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-872cm"
Jul 10 02:39:52.921202 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.921175    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qqvdz"
Jul 10 02:39:52.926211 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:52.926189    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:39:53.225951 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.225854    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:53.225951 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.225910    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:53.226163 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:53.226044    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:39:53.226163 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:53.226074    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:39:53.226163 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:53.226137    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret podName:3d813814-8b23-41d4-852b-ff842e8a5771 nodeName:}" failed. No retries permitted until 2026-07-10 02:39:54.226114787 +0000 UTC m=+4.035380093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret") pod "global-pull-secret-syncer-npnft" (UID: "3d813814-8b23-41d4-852b-ff842e8a5771") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:39:53.226163 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:53.226159    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs podName:cb9fe830-29c0-4665-beea-ffda1ca5ef53 nodeName:}" failed. No retries permitted until 2026-07-10 02:39:54.226148507 +0000 UTC m=+4.035413792 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs") pod "network-metrics-daemon-57zjp" (UID: "cb9fe830-29c0-4665-beea-ffda1ca5ef53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:39:53.326968 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.326926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77zn\" (UniqueName: \"kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn\") pod \"network-check-target-q4q57\" (UID: \"89160553-786f-443a-8e3b-d6ffb4f4b761\") " pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:39:53.327189 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:53.327099    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:39:53.327189 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:53.327123    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:39:53.327189 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:53.327135    2576 projected.go:196] Error preparing data for projected volume kube-api-access-w77zn for pod openshift-network-diagnostics/network-check-target-q4q57: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:39:53.327368 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:53.327206    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn podName:89160553-786f-443a-8e3b-d6ffb4f4b761 nodeName:}" failed. No retries permitted until 2026-07-10 02:39:54.327184616 +0000 UTC m=+4.136449912 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w77zn" (UniqueName: "kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn") pod "network-check-target-q4q57" (UID: "89160553-786f-443a-8e3b-d6ffb4f4b761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:39:53.610310 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.610207    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:34:53 +0000 UTC" deadline="2028-04-08 16:04:35.044742298 +0000 UTC"
Jul 10 02:39:53.610310 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.610262    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15325h24m41.43448438s"
Jul 10 02:39:53.805779 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.805713    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k2s62" event={"ID":"09ab858c-346a-4ccd-9834-627796c08d79","Type":"ContainerStarted","Data":"f4b0a4395bf9927b920688081017a6f167a2ececf0d0866eeba058d390066664"}
Jul 10 02:39:53.807976 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.807938    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" event={"ID":"61046ad7-86b4-4781-9728-e7710e740f72","Type":"ContainerStarted","Data":"aa305232788c49b0d114a3cd3a946ebf3f4ccfdd2caedcc8f2ae72be2f3aaab9"}
Jul 10 02:39:53.809816 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.809789    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r7k5d" event={"ID":"01980251-990d-4898-b259-8a05b74de2fc","Type":"ContainerStarted","Data":"de2d48d68ca768a24644f523affc142bfb4c18fa79fc145cc58c107537da7151"}
Jul 10 02:39:53.814370 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.814330    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmb92" event={"ID":"228cc28c-9b29-4a85-a465-040c721653fa","Type":"ContainerStarted","Data":"2b78fd37230d6487398d25233fffc06cb000a8598890e643f8c3358a672c70b8"}
Jul 10 02:39:53.816174 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.816147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8" event={"ID":"f6649dea-7a4b-435f-9696-ef409686f070","Type":"ContainerStarted","Data":"dcc393a6926208d4dbd1fc69695ce6a2b265b8a0ce3d9ce5e2f7037f879b9ef4"}
Jul 10 02:39:53.819219 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:53.819193    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cfmzm" event={"ID":"22008c3f-7d3c-4151-910c-8950c916a84c","Type":"ContainerStarted","Data":"4219d63050a79b303936da7e9d85917d9b9305d50b22d2208a88ba11402dd3eb"}
Jul 10 02:39:54.201118 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:54.201013    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6adfb3eb_939c_4bd9_8a87_2e66b89b9ece.slice/crio-755c2515f94f668512a92f5f241a426fa464bdae541df192923c510abd7802c9 WatchSource:0}: Error finding container 755c2515f94f668512a92f5f241a426fa464bdae541df192923c510abd7802c9: Status 404 returned error can't find the container with id 755c2515f94f668512a92f5f241a426fa464bdae541df192923c510abd7802c9
Jul 10 02:39:54.209156 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:39:54.209109    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0179f78_2a2a_4f3f_bbd3_bcb655dd6e10.slice/crio-0fd51bbcdb590010295708aa4a8d534238f5c1f0c415fb2ecde03d2968592ae0 WatchSource:0}: Error finding container 0fd51bbcdb590010295708aa4a8d534238f5c1f0c415fb2ecde03d2968592ae0: Status 404 returned error can't find the container with id 0fd51bbcdb590010295708aa4a8d534238f5c1f0c415fb2ecde03d2968592ae0
Jul 10 02:39:54.234687 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.234615    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:54.234687 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.234663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:54.234905 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.234860    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:39:54.234964 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.234931    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs podName:cb9fe830-29c0-4665-beea-ffda1ca5ef53 nodeName:}" failed. No retries permitted until 2026-07-10 02:39:56.234906965 +0000 UTC m=+6.044172273 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs") pod "network-metrics-daemon-57zjp" (UID: "cb9fe830-29c0-4665-beea-ffda1ca5ef53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:39:54.235350 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.235302    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:39:54.235438 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.235362    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret podName:3d813814-8b23-41d4-852b-ff842e8a5771 nodeName:}" failed. No retries permitted until 2026-07-10 02:39:56.235343672 +0000 UTC m=+6.044608971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret") pod "global-pull-secret-syncer-npnft" (UID: "3d813814-8b23-41d4-852b-ff842e8a5771") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:39:54.336342 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.336063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77zn\" (UniqueName: \"kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn\") pod \"network-check-target-q4q57\" (UID: \"89160553-786f-443a-8e3b-d6ffb4f4b761\") " pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:39:54.336596 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.336365    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:39:54.336596 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.336397    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:39:54.336596 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.336411    2576 projected.go:196] Error preparing data for projected volume kube-api-access-w77zn for pod openshift-network-diagnostics/network-check-target-q4q57: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:39:54.336596 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.336483    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn podName:89160553-786f-443a-8e3b-d6ffb4f4b761 nodeName:}" failed. No retries permitted until 2026-07-10 02:39:56.336463253 +0000 UTC m=+6.145728549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w77zn" (UniqueName: "kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn") pod "network-check-target-q4q57" (UID: "89160553-786f-443a-8e3b-d6ffb4f4b761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:39:54.612673 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.611227    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:34:53 +0000 UTC" deadline="2028-01-11 02:07:00.22030699 +0000 UTC"
Jul 10 02:39:54.612673 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.611269    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13199h27m5.609042564s"
Jul 10 02:39:54.799173 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.797985    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:54.799173 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.798214    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:39:54.799173 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.798334    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:39:54.799173 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.798461    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:39:54.799173 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.798961    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:54.799173 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:54.799119    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:39:54.823848 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.823743    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-872cm" event={"ID":"528c8356-c22a-40a3-a768-7a7c90c6c44f","Type":"ContainerStarted","Data":"5addaa574404247c79fe0e78f358bd0a8a4f373b4f15c8276d373b25c4dc1e50"}
Jul 10 02:39:54.832870 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.832146    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal" event={"ID":"7267bfa133f7ae1ca9f615a6770bb5df","Type":"ContainerStarted","Data":"d0ad08aa29a329714c8084f5953f55e6c99f88d6aa58984f47c6c8f53c956c73"}
Jul 10 02:39:54.843879 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.843456    2576 generic.go:356] "Generic (PLEG): container finished" podID="75342c7e3311385e2be3c26bd0e39bb2" containerID="9cc46771cbb07da559409d5e6074b41c8ce658c35aa7a07c32959054c2c0e6f9" exitCode=0
Jul 10 02:39:54.843879 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.843544    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal" event={"ID":"75342c7e3311385e2be3c26bd0e39bb2","Type":"ContainerDied","Data":"9cc46771cbb07da559409d5e6074b41c8ce658c35aa7a07c32959054c2c0e6f9"}
Jul 10 02:39:54.863804 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.863711    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gphcf" event={"ID":"a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10","Type":"ContainerStarted","Data":"0fd51bbcdb590010295708aa4a8d534238f5c1f0c415fb2ecde03d2968592ae0"}
Jul 10 02:39:54.866319 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.866258    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-93.ec2.internal" podStartSLOduration=2.866237452 podStartE2EDuration="2.866237452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:39:54.850121044 +0000 UTC m=+4.659386352" watchObservedRunningTime="2026-07-10 02:39:54.866237452 +0000 UTC m=+4.675502760"
Jul 10 02:39:54.877194 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:54.877153    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qqvdz" event={"ID":"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece","Type":"ContainerStarted","Data":"755c2515f94f668512a92f5f241a426fa464bdae541df192923c510abd7802c9"}
Jul 10 02:39:55.891339 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:55.891299    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal" event={"ID":"75342c7e3311385e2be3c26bd0e39bb2","Type":"ContainerStarted","Data":"7756faebba64cc069b1030b720e0e98125b1c43ff438d02842a20d987aed1d38"}
Jul 10 02:39:56.253275 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:56.252369    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:56.253275 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:56.252430    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:56.253275 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.252594    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:39:56.253275 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.252664    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs podName:cb9fe830-29c0-4665-beea-ffda1ca5ef53 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:00.252644246 +0000 UTC m=+10.061909544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs") pod "network-metrics-daemon-57zjp" (UID: "cb9fe830-29c0-4665-beea-ffda1ca5ef53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:39:56.253275 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.253117    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:39:56.253275 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.253174    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret podName:3d813814-8b23-41d4-852b-ff842e8a5771 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:00.253156882 +0000 UTC m=+10.062422167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret") pod "global-pull-secret-syncer-npnft" (UID: "3d813814-8b23-41d4-852b-ff842e8a5771") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:39:56.353385 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:56.353341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77zn\" (UniqueName: \"kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn\") pod \"network-check-target-q4q57\" (UID: \"89160553-786f-443a-8e3b-d6ffb4f4b761\") " pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:39:56.353572 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.353530    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:39:56.353572 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.353551    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:39:56.353572 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.353564    2576 projected.go:196] Error preparing data for projected volume kube-api-access-w77zn for pod openshift-network-diagnostics/network-check-target-q4q57: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:39:56.353767 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.353628    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn podName:89160553-786f-443a-8e3b-d6ffb4f4b761 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:00.353608963 +0000 UTC m=+10.162874249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w77zn" (UniqueName: "kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn") pod "network-check-target-q4q57" (UID: "89160553-786f-443a-8e3b-d6ffb4f4b761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:39:56.793016 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:56.792945    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:56.793016 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:56.792966    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:56.793267 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.793105    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:39:56.793267 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.793215    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:39:56.793267 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:56.793256    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:39:56.793427 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:56.793327    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:39:58.794473 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:58.793577    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:39:58.794473 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:58.793595    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:39:58.794473 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:58.793725    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:39:58.794473 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:58.794220    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:39:58.794473 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:39:58.794278    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:39:58.794473 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:39:58.794389    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:00.290113 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:00.290068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:00.290594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:00.290130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:00.290594 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.290260    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:40:00.290594 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.290276    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:40:00.290594 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.290337    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret podName:3d813814-8b23-41d4-852b-ff842e8a5771 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:08.290316603 +0000 UTC m=+18.099581908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret") pod "global-pull-secret-syncer-npnft" (UID: "3d813814-8b23-41d4-852b-ff842e8a5771") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:40:00.290594 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.290359    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs podName:cb9fe830-29c0-4665-beea-ffda1ca5ef53 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:08.290348831 +0000 UTC m=+18.099614111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs") pod "network-metrics-daemon-57zjp" (UID: "cb9fe830-29c0-4665-beea-ffda1ca5ef53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:40:00.391498 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:00.390963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77zn\" (UniqueName: \"kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn\") pod \"network-check-target-q4q57\" (UID: \"89160553-786f-443a-8e3b-d6ffb4f4b761\") " pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:00.391498 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.391058    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:40:00.391498 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.391078    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:40:00.391498 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.391091    2576 projected.go:196] Error preparing data for projected volume kube-api-access-w77zn for pod openshift-network-diagnostics/network-check-target-q4q57: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:40:00.391498 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.391150    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn podName:89160553-786f-443a-8e3b-d6ffb4f4b761 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:08.391131503 +0000 UTC m=+18.200396806 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w77zn" (UniqueName: "kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn") pod "network-check-target-q4q57" (UID: "89160553-786f-443a-8e3b-d6ffb4f4b761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:40:00.793460 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:00.793428    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:00.793642 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.793537    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:00.793642 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:00.793608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:00.793770 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:00.793734    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:00.793829 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.793760    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:00.793932 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:00.793877    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:02.793492 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:02.793450    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:02.793947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:02.793510    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:02.793947 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:02.793633    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:02.794067 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:02.794008    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:02.794162 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:02.794120    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:02.794238 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:02.794218    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:04.793210 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:04.793177    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:04.793210 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:04.793197    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:04.793730 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:04.793213    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:04.793730 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:04.793306    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:04.793730 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:04.793381    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:04.793730 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:04.793505    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:06.793414 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:06.793383    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:06.793414 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:06.793391    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:06.793414 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:06.793420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:06.793939 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:06.793511    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:06.793939 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:06.793653    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:06.793939 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:06.793723    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:08.349076 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:08.349038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:08.349076 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:08.349080    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:08.349591 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.349204    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:40:08.349591 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.349261    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs podName:cb9fe830-29c0-4665-beea-ffda1ca5ef53 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:24.349246559 +0000 UTC m=+34.158511840 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs") pod "network-metrics-daemon-57zjp" (UID: "cb9fe830-29c0-4665-beea-ffda1ca5ef53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:40:08.349591 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.349204    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:40:08.349591 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.349354    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret podName:3d813814-8b23-41d4-852b-ff842e8a5771 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:24.349335081 +0000 UTC m=+34.158600383 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret") pod "global-pull-secret-syncer-npnft" (UID: "3d813814-8b23-41d4-852b-ff842e8a5771") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:40:08.449903 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:08.449858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77zn\" (UniqueName: \"kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn\") pod \"network-check-target-q4q57\" (UID: \"89160553-786f-443a-8e3b-d6ffb4f4b761\") " pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:08.450068 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.450037    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:40:08.450068 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.450057    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:40:08.450068 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.450067    2576 projected.go:196] Error preparing data for projected volume kube-api-access-w77zn for pod openshift-network-diagnostics/network-check-target-q4q57: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:40:08.450236 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.450135    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn podName:89160553-786f-443a-8e3b-d6ffb4f4b761 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:24.450116079 +0000 UTC m=+34.259381385 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w77zn" (UniqueName: "kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn") pod "network-check-target-q4q57" (UID: "89160553-786f-443a-8e3b-d6ffb4f4b761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:40:08.792862 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:08.792824    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:08.792862 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:08.792848    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:08.793118 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:08.792850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:08.793118 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.792953    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:08.793118 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.793048    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:08.793305 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:08.793132    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:10.794118 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:10.794071    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:10.794602 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:10.794178    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:10.794602 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:10.794190    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:10.794602 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:10.794293    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:10.794602 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:10.794322    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:10.794602 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:10.794386    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:12.793032 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.792791    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:12.793556 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.792791    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:12.793556 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:12.793134    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:12.793556 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.792791    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:12.793556 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:12.793243    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:12.793556 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:12.793272    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:12.924442 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.924415    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k2s62" event={"ID":"09ab858c-346a-4ccd-9834-627796c08d79","Type":"ContainerStarted","Data":"7f1e14be41dd2261484b3a778c2b02b8b2d78cf6efc21fef94450673d245e2c8"}
Jul 10 02:40:12.926741 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.926722    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:40:12.927081 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.927059    2576 generic.go:356] "Generic (PLEG): container finished" podID="61046ad7-86b4-4781-9728-e7710e740f72" containerID="5588628fa93ffe02c5f4f23bf693d1740c6db0448f86db26cea31342c6b1907f" exitCode=1
Jul 10 02:40:12.927151 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.927129    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" event={"ID":"61046ad7-86b4-4781-9728-e7710e740f72","Type":"ContainerStarted","Data":"3a2b4d59e0aab3cd8f463c7421f797b93dc68ad9a37e4b5481a2910a2456b9eb"}
Jul 10 02:40:12.927236 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.927223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" event={"ID":"61046ad7-86b4-4781-9728-e7710e740f72","Type":"ContainerStarted","Data":"21661e1b6b10499431a0b3e14a630c8d5820fb65bf915cc1fcb00f02f42a142b"}
Jul 10 02:40:12.927292 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.927275    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" event={"ID":"61046ad7-86b4-4781-9728-e7710e740f72","Type":"ContainerStarted","Data":"60c57d696233c1dffdc389e58846a294284e0401d10b491d897b403c0a8f8337"}
Jul 10 02:40:12.927349 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.927325    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" event={"ID":"61046ad7-86b4-4781-9728-e7710e740f72","Type":"ContainerDied","Data":"5588628fa93ffe02c5f4f23bf693d1740c6db0448f86db26cea31342c6b1907f"}
Jul 10 02:40:12.927402 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.927376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" event={"ID":"61046ad7-86b4-4781-9728-e7710e740f72","Type":"ContainerStarted","Data":"40d3c58b41414a687e0acce90db167a3fcd7a762e8db206a5db30c918c38fbf5"}
Jul 10 02:40:12.928509 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.928485    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gphcf" event={"ID":"a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10","Type":"ContainerStarted","Data":"37f3cd80b688b15e4f01932ccd3ee5985239a39664ca9b368449a0d3159f8505"}
Jul 10 02:40:12.929916 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.929889    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qqvdz" event={"ID":"6adfb3eb-939c-4bd9-8a87-2e66b89b9ece","Type":"ContainerStarted","Data":"64a175f8ac57a9c3c10f5c7878f8ce3a869815da53c0902aeb9679f3c172d7ea"}
Jul 10 02:40:12.931689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.931665    2576 generic.go:356] "Generic (PLEG): container finished" podID="228cc28c-9b29-4a85-a465-040c721653fa" containerID="c10c5267d2f6d2d7e8308fce79fdcda27ea7c3c6316906dc2b02541617b39d9a" exitCode=0
Jul 10 02:40:12.931793 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.931733    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmb92" event={"ID":"228cc28c-9b29-4a85-a465-040c721653fa","Type":"ContainerDied","Data":"c10c5267d2f6d2d7e8308fce79fdcda27ea7c3c6316906dc2b02541617b39d9a"}
Jul 10 02:40:12.933289 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.933228    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8" event={"ID":"f6649dea-7a4b-435f-9696-ef409686f070","Type":"ContainerStarted","Data":"b48a0c5a33f006af0ad6ee7aad73d7be81a1bd2d5c6775a8a07b4732aec11f54"}
Jul 10 02:40:12.934646 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.934618    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cfmzm" event={"ID":"22008c3f-7d3c-4151-910c-8950c916a84c","Type":"ContainerStarted","Data":"d4bc4546d3d2a1d698ec9fe661e2796c3fc4e89d06baefca1cf53677b5509c66"}
Jul 10 02:40:12.936106 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.936083    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r7k5d" event={"ID":"01980251-990d-4898-b259-8a05b74de2fc","Type":"ContainerStarted","Data":"4155126f78195ab378850d4296c92bc8ae5d6bc7764539b689136fcd608d918d"}
Jul 10 02:40:12.941925 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.941873    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k2s62" podStartSLOduration=2.468161989 podStartE2EDuration="21.941857173s" totalImagesPullingTime="19.473695184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:12.941502034 +0000 UTC m=+22.750767333" watchObservedRunningTime="2026-07-10 02:40:12.941857173 +0000 UTC m=+22.751122478"
Jul 10 02:40:12.942257 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.942222    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-93.ec2.internal" podStartSLOduration=20.942209854 podStartE2EDuration="20.942209854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:39:55.908231216 +0000 UTC m=+5.717496526" watchObservedRunningTime="2026-07-10 02:40:12.942209854 +0000 UTC m=+22.751475156"
Jul 10 02:40:12.954689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.954647    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r7k5d" podStartSLOduration=2.5225607759999997 podStartE2EDuration="21.954633453s" totalImagesPullingTime="19.432072677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:12.954363279 +0000 UTC m=+22.763628574" watchObservedRunningTime="2026-07-10 02:40:12.954633453 +0000 UTC m=+22.763898755"
Jul 10 02:40:12.972806 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.972739    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qqvdz" podStartSLOduration=3.870662251 podStartE2EDuration="21.972724174s" totalImagesPullingTime="18.102061923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:12.972196072 +0000 UTC m=+22.781461373" watchObservedRunningTime="2026-07-10 02:40:12.972724174 +0000 UTC m=+22.781989476"
Jul 10 02:40:12.992341 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:12.992290    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gphcf" podStartSLOduration=3.932020219 podStartE2EDuration="21.992276651s" totalImagesPullingTime="18.060256432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:12.991851676 +0000 UTC m=+22.801116977" watchObservedRunningTime="2026-07-10 02:40:12.992276651 +0000 UTC m=+22.801541952"
Jul 10 02:40:13.009510 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:13.009462    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cfmzm" podStartSLOduration=2.626459499 podStartE2EDuration="22.009447573s" totalImagesPullingTime="19.382988074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:13.009091353 +0000 UTC m=+22.818356658" watchObservedRunningTime="2026-07-10 02:40:13.009447573 +0000 UTC m=+22.818712874"
Jul 10 02:40:13.941884 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:13.941852    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:40:13.942341 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:13.942308    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" event={"ID":"61046ad7-86b4-4781-9728-e7710e740f72","Type":"ContainerStarted","Data":"eb29083573efbd5365a258baca614e203f39e599e373fb388a9f528dea597707"}
Jul 10 02:40:13.943945 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:13.943890    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-872cm" event={"ID":"528c8356-c22a-40a3-a768-7a7c90c6c44f","Type":"ContainerStarted","Data":"b0c59c2f3f8433360c72be3ced878660c13bcffdb541f60a328193c5da2e1a56"}
Jul 10 02:40:13.957283 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:13.957233    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-872cm" podStartSLOduration=4.888867955 podStartE2EDuration="22.957212224s" totalImagesPullingTime="18.068344269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:13.956972947 +0000 UTC m=+23.766238254" watchObservedRunningTime="2026-07-10 02:40:13.957212224 +0000 UTC m=+23.766477527"
Jul 10 02:40:14.005175 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:14.005148    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 02:40:14.658716 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:14.658559    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T02:40:14.005170421Z","UUID":"eb551a49-104d-4ae8-a4e8-eb042b88e5b4","Handler":null,"Name":"","Endpoint":""}
Jul 10 02:40:14.660039 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:14.660014    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 02:40:14.660169 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:14.660053    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 02:40:14.793461 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:14.793419    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:14.793461 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:14.793444    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:14.793696 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:14.793425    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:14.793696 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:14.793553    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:14.793696 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:14.793665    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:14.793866 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:14.793770    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:14.948607 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:14.948516    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8" event={"ID":"f6649dea-7a4b-435f-9696-ef409686f070","Type":"ContainerStarted","Data":"e30e26d6c40f19caf3a0fb37bd7650a01feaa202682c269b26f105d516880299"}
Jul 10 02:40:15.500933 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:15.500885    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:40:15.501592 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:15.501571    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:40:15.953480 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:15.953173    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8" event={"ID":"f6649dea-7a4b-435f-9696-ef409686f070","Type":"ContainerStarted","Data":"ae3c3231c0f04d3b536bfd564faba824bd97a2442931ffc0a9b9f3eed8314ad3"}
Jul 10 02:40:15.956763 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:15.956727    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:40:15.957241 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:15.957202    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" event={"ID":"61046ad7-86b4-4781-9728-e7710e740f72","Type":"ContainerStarted","Data":"07a0bbd2a335390649ae73b3476c9a4e3cc2924bc514ca628f5c7787199cbd57"}
Jul 10 02:40:15.957495 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:15.957472    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:40:15.958083 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:15.958065    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gphcf"
Jul 10 02:40:15.970541 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:15.970496    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5ql8" podStartSLOduration=2.683238576 podStartE2EDuration="24.970480979s" totalImagesPullingTime="22.287242403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:15.970124527 +0000 UTC m=+25.779389834" watchObservedRunningTime="2026-07-10 02:40:15.970480979 +0000 UTC m=+25.779746291"
Jul 10 02:40:16.793314 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:16.793268    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:16.793314 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:16.793295    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:16.793643 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:16.793421    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:16.793643 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:16.793475    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:16.793643 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:16.793554    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:16.793787 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:16.793642    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:17.964964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:17.964776    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:40:17.965717 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:17.965363    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" event={"ID":"61046ad7-86b4-4781-9728-e7710e740f72","Type":"ContainerStarted","Data":"2f487b179be4fa2cff01ba2717e6605ec9ff0c7caf29dee7599e3a1157ee111e"}
Jul 10 02:40:17.965717 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:17.965693    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:40:17.965831 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:17.965787    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:40:17.965901 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:17.965834    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:40:17.965901 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:17.965882    2576 scope.go:117] "RemoveContainer" containerID="5588628fa93ffe02c5f4f23bf693d1740c6db0448f86db26cea31342c6b1907f"
Jul 10 02:40:17.984168 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:17.984143    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:40:17.984326 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:17.984264    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:40:18.794178 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:18.794089    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:18.794367 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:18.794225    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:18.794679 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:18.794508    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:18.794679 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:18.794628    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:18.794850 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:18.794727    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:18.796319 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:18.794933    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:19.524352 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:19.523922    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-npnft"]
Jul 10 02:40:19.524352 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:19.524022    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:19.524352 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:19.524128    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:19.526915 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:19.526818    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q4q57"]
Jul 10 02:40:19.526915 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:19.526916    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:19.527129 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:19.527006    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:19.536409 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:19.536384    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-57zjp"]
Jul 10 02:40:19.536535 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:19.536486    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:19.536593 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:19.536574    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:19.974312 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:19.974284    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:40:19.974706 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:19.974679    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" event={"ID":"61046ad7-86b4-4781-9728-e7710e740f72","Type":"ContainerStarted","Data":"ac0f47d964236aff37167a66766c8a496e1c01c776763a762063e564b9b57694"}
Jul 10 02:40:19.976480 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:19.976450    2576 generic.go:356] "Generic (PLEG): container finished" podID="228cc28c-9b29-4a85-a465-040c721653fa" containerID="5720001a75da6fbaa7310ad0527045953c76ad6d951ae1c8de9920083815c022" exitCode=0
Jul 10 02:40:19.976591 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:19.976501    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmb92" event={"ID":"228cc28c-9b29-4a85-a465-040c721653fa","Type":"ContainerDied","Data":"5720001a75da6fbaa7310ad0527045953c76ad6d951ae1c8de9920083815c022"}
Jul 10 02:40:20.001457 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:20.001388    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg" podStartSLOduration=9.485701362 podStartE2EDuration="29.001368382s" totalImagesPullingTime="19.51566702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:20.001128671 +0000 UTC m=+29.810393994" watchObservedRunningTime="2026-07-10 02:40:20.001368382 +0000 UTC m=+29.810633685"
Jul 10 02:40:21.793663 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:21.793480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:21.794083 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:21.793480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:21.794083 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:21.793765    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:21.794083 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:21.793480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:21.794083 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:21.793829    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:21.794083 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:21.793933    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:21.983051 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:21.983020    2576 generic.go:356] "Generic (PLEG): container finished" podID="228cc28c-9b29-4a85-a465-040c721653fa" containerID="af8abb79dc4a085ccd10ed498e53c5fcb93730bb1cf4bf398c844780f48743a3" exitCode=0
Jul 10 02:40:21.983179 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:21.983073    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmb92" event={"ID":"228cc28c-9b29-4a85-a465-040c721653fa","Type":"ContainerDied","Data":"af8abb79dc4a085ccd10ed498e53c5fcb93730bb1cf4bf398c844780f48743a3"}
Jul 10 02:40:22.988238 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:22.988198    2576 generic.go:356] "Generic (PLEG): container finished" podID="228cc28c-9b29-4a85-a465-040c721653fa" containerID="913874bae76e363563762461b2cbc656a3b3f9904506a4798b16326272d42b82" exitCode=0
Jul 10 02:40:22.988928 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:22.988260    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmb92" event={"ID":"228cc28c-9b29-4a85-a465-040c721653fa","Type":"ContainerDied","Data":"913874bae76e363563762461b2cbc656a3b3f9904506a4798b16326272d42b82"}
Jul 10 02:40:23.793526 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:23.793483    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:23.793526 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:23.793524    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:23.793796 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:23.793525    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:23.793796 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:23.793600    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57zjp" podUID="cb9fe830-29c0-4665-beea-ffda1ca5ef53"
Jul 10 02:40:23.793796 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:23.793706    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npnft" podUID="3d813814-8b23-41d4-852b-ff842e8a5771"
Jul 10 02:40:23.793924 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:23.793820    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q4q57" podUID="89160553-786f-443a-8e3b-d6ffb4f4b761"
Jul 10 02:40:24.362271 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:24.362240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:24.362678 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:24.362279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:24.362678 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:24.362391    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:40:24.362678 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:24.362399    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:40:24.362678 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:24.362448    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs podName:cb9fe830-29c0-4665-beea-ffda1ca5ef53 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:56.362433907 +0000 UTC m=+66.171699187 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs") pod "network-metrics-daemon-57zjp" (UID: "cb9fe830-29c0-4665-beea-ffda1ca5ef53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:40:24.362678 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:24.362462    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret podName:3d813814-8b23-41d4-852b-ff842e8a5771 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:56.362455455 +0000 UTC m=+66.171720735 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret") pod "global-pull-secret-syncer-npnft" (UID: "3d813814-8b23-41d4-852b-ff842e8a5771") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:40:24.463331 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:24.463302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77zn\" (UniqueName: \"kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn\") pod \"network-check-target-q4q57\" (UID: \"89160553-786f-443a-8e3b-d6ffb4f4b761\") " pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:24.463492 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:24.463444    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:40:24.463492 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:24.463461    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:40:24.463492 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:24.463471    2576 projected.go:196] Error preparing data for projected volume kube-api-access-w77zn for pod openshift-network-diagnostics/network-check-target-q4q57: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:40:24.463658 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:24.463530    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn podName:89160553-786f-443a-8e3b-d6ffb4f4b761 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:56.463511752 +0000 UTC m=+66.272777046 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w77zn" (UniqueName: "kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn") pod "network-check-target-q4q57" (UID: "89160553-786f-443a-8e3b-d6ffb4f4b761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:40:24.968190 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:24.968166    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-93.ec2.internal" event="NodeReady"
Jul 10 02:40:24.968348 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:24.968271    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 02:40:25.002994 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.002962    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-df94489-vlxg7"]
Jul 10 02:40:25.005201 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.005178    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.007896 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.007829    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 02:40:25.007896 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.007842    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 02:40:25.008061 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.007908    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 02:40:25.008061 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.007914    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w6ngk\""
Jul 10 02:40:25.016824 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.016794    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 02:40:25.023553 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.023524    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-df94489-vlxg7"]
Jul 10 02:40:25.024377 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.024351    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-979w2"]
Jul 10 02:40:25.027232 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.027213    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fktxn"]
Jul 10 02:40:25.027355 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.027341    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.029255 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.029235    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fktxn"
Jul 10 02:40:25.029925 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.029906    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 02:40:25.030014 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.029941    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 02:40:25.030014 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.030002    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8l52l\""
Jul 10 02:40:25.031610 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.031590    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 02:40:25.031710 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.031644    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 02:40:25.031836 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.031808    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 02:40:25.031931 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.031915    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2tgb9\""
Jul 10 02:40:25.037830 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.037810    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-979w2"]
Jul 10 02:40:25.040810 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.040790    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fktxn"]
Jul 10 02:40:25.067554 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067463    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-image-registry-private-configuration\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.067554 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067507    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b15416cc-0665-46f9-af55-f1eb77451bbc-tmp-dir\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.067554 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067538    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c7dab0-cc72-4598-8761-2256c99bb318-cert\") pod \"ingress-canary-fktxn\" (UID: \"b5c7dab0-cc72-4598-8761-2256c99bb318\") " pod="openshift-ingress-canary/ingress-canary-fktxn"
Jul 10 02:40:25.067776 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067563    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.067776 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ddb8f374-e401-461b-8618-921fa7187864-ca-trust-extracted\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.067776 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067709    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b15416cc-0665-46f9-af55-f1eb77451bbc-config-volume\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.067776 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067734    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.067776 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067771    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-registry-certificates\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.068014 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-installation-pull-secrets\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.068014 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067841    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-bound-sa-token\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.068014 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067861    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzrb4\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-kube-api-access-fzrb4\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.068014 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-trusted-ca\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.068014 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t22fq\" (UniqueName: \"kubernetes.io/projected/b15416cc-0665-46f9-af55-f1eb77451bbc-kube-api-access-t22fq\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.068014 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.067960    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcbpw\" (UniqueName: \"kubernetes.io/projected/b5c7dab0-cc72-4598-8761-2256c99bb318-kube-api-access-mcbpw\") pod \"ingress-canary-fktxn\" (UID: \"b5c7dab0-cc72-4598-8761-2256c99bb318\") " pod="openshift-ingress-canary/ingress-canary-fktxn"
Jul 10 02:40:25.168554 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168510    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b15416cc-0665-46f9-af55-f1eb77451bbc-config-volume\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.168554 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168549    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.168815 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-registry-certificates\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.168815 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168596    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-installation-pull-secrets\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.168815 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:25.168688    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:40:25.168815 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-bound-sa-token\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.168815 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:25.168781    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls podName:b15416cc-0665-46f9-af55-f1eb77451bbc nodeName:}" failed. No retries permitted until 2026-07-10 02:40:25.66874128 +0000 UTC m=+35.478006565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls") pod "dns-default-979w2" (UID: "b15416cc-0665-46f9-af55-f1eb77451bbc") : secret "dns-default-metrics-tls" not found
Jul 10 02:40:25.168815 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzrb4\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-kube-api-access-fzrb4\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.169125 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-trusted-ca\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.169125 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168881    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t22fq\" (UniqueName: \"kubernetes.io/projected/b15416cc-0665-46f9-af55-f1eb77451bbc-kube-api-access-t22fq\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.169125 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcbpw\" (UniqueName: \"kubernetes.io/projected/b5c7dab0-cc72-4598-8761-2256c99bb318-kube-api-access-mcbpw\") pod \"ingress-canary-fktxn\" (UID: \"b5c7dab0-cc72-4598-8761-2256c99bb318\") " pod="openshift-ingress-canary/ingress-canary-fktxn"
Jul 10 02:40:25.169125 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-image-registry-private-configuration\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.169125 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.168988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b15416cc-0665-46f9-af55-f1eb77451bbc-tmp-dir\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.169125 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.169015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c7dab0-cc72-4598-8761-2256c99bb318-cert\") pod \"ingress-canary-fktxn\" (UID: \"b5c7dab0-cc72-4598-8761-2256c99bb318\") " pod="openshift-ingress-canary/ingress-canary-fktxn"
Jul 10 02:40:25.169403 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.169256    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-registry-certificates\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.169403 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.169265    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b15416cc-0665-46f9-af55-f1eb77451bbc-config-volume\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.169403 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.169379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b15416cc-0665-46f9-af55-f1eb77451bbc-tmp-dir\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.169403 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.169384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.169586 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.169458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ddb8f374-e401-461b-8618-921fa7187864-ca-trust-extracted\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.169586 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:25.169477    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:40:25.169586 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:25.169498    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df94489-vlxg7: secret "image-registry-tls" not found
Jul 10 02:40:25.169586 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:25.169556    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls podName:ddb8f374-e401-461b-8618-921fa7187864 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:25.669539604 +0000 UTC m=+35.478804899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls") pod "image-registry-df94489-vlxg7" (UID: "ddb8f374-e401-461b-8618-921fa7187864") : secret "image-registry-tls" not found
Jul 10 02:40:25.169792 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.169770    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ddb8f374-e401-461b-8618-921fa7187864-ca-trust-extracted\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.170857 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.170832    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-trusted-ca\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.173439 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.173413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-image-registry-private-configuration\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.173536 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.173440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-installation-pull-secrets\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.173536 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.173494    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c7dab0-cc72-4598-8761-2256c99bb318-cert\") pod \"ingress-canary-fktxn\" (UID: \"b5c7dab0-cc72-4598-8761-2256c99bb318\") " pod="openshift-ingress-canary/ingress-canary-fktxn"
Jul 10 02:40:25.177830 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.177805    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcbpw\" (UniqueName: \"kubernetes.io/projected/b5c7dab0-cc72-4598-8761-2256c99bb318-kube-api-access-mcbpw\") pod \"ingress-canary-fktxn\" (UID: \"b5c7dab0-cc72-4598-8761-2256c99bb318\") " pod="openshift-ingress-canary/ingress-canary-fktxn"
Jul 10 02:40:25.178108 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.178086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-bound-sa-token\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.178380 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.178359    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t22fq\" (UniqueName: \"kubernetes.io/projected/b15416cc-0665-46f9-af55-f1eb77451bbc-kube-api-access-t22fq\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.179053 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.179029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzrb4\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-kube-api-access-fzrb4\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.347454 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.347369    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fktxn"
Jul 10 02:40:25.499903 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.499652    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fktxn"]
Jul 10 02:40:25.504313 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:40:25.504279    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c7dab0_cc72_4598_8761_2256c99bb318.slice/crio-b51582040ba802ded27d657dafc1d4d6a3cecdd6a314cb4ab54f9233ff17ac9d WatchSource:0}: Error finding container b51582040ba802ded27d657dafc1d4d6a3cecdd6a314cb4ab54f9233ff17ac9d: Status 404 returned error can't find the container with id b51582040ba802ded27d657dafc1d4d6a3cecdd6a314cb4ab54f9233ff17ac9d
Jul 10 02:40:25.673210 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.673100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:25.673210 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.673164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:25.673412 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:25.673266    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:40:25.673412 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:25.673337    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls podName:b15416cc-0665-46f9-af55-f1eb77451bbc nodeName:}" failed. No retries permitted until 2026-07-10 02:40:26.673317714 +0000 UTC m=+36.482583017 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls") pod "dns-default-979w2" (UID: "b15416cc-0665-46f9-af55-f1eb77451bbc") : secret "dns-default-metrics-tls" not found
Jul 10 02:40:25.673412 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:25.673267    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:40:25.673412 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:25.673362    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df94489-vlxg7: secret "image-registry-tls" not found
Jul 10 02:40:25.673412 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:25.673397    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls podName:ddb8f374-e401-461b-8618-921fa7187864 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:26.673387224 +0000 UTC m=+36.482652519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls") pod "image-registry-df94489-vlxg7" (UID: "ddb8f374-e401-461b-8618-921fa7187864") : secret "image-registry-tls" not found
Jul 10 02:40:25.793048 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.793004    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:25.793048 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.793026    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:25.793048 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.793031    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:25.795803 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.795782    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 02:40:25.796127 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.796107    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jw76v\""
Jul 10 02:40:25.796178 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.796123    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 02:40:25.796178 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.796115    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9pgv\""
Jul 10 02:40:25.796465 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.796446    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 02:40:25.796465 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.796453    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 02:40:25.998407 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:25.998369    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fktxn" event={"ID":"b5c7dab0-cc72-4598-8761-2256c99bb318","Type":"ContainerStarted","Data":"b51582040ba802ded27d657dafc1d4d6a3cecdd6a314cb4ab54f9233ff17ac9d"}
Jul 10 02:40:26.681712 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:26.681672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:26.682232 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:26.681766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:26.682232 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:26.681929    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:40:26.682232 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:26.681992    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:40:26.682232 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:26.682011    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls podName:b15416cc-0665-46f9-af55-f1eb77451bbc nodeName:}" failed. No retries permitted until 2026-07-10 02:40:28.681988339 +0000 UTC m=+38.491253622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls") pod "dns-default-979w2" (UID: "b15416cc-0665-46f9-af55-f1eb77451bbc") : secret "dns-default-metrics-tls" not found
Jul 10 02:40:26.682232 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:26.682014    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df94489-vlxg7: secret "image-registry-tls" not found
Jul 10 02:40:26.682232 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:26.682072    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls podName:ddb8f374-e401-461b-8618-921fa7187864 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:28.682054029 +0000 UTC m=+38.491319325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls") pod "image-registry-df94489-vlxg7" (UID: "ddb8f374-e401-461b-8618-921fa7187864") : secret "image-registry-tls" not found
Jul 10 02:40:28.698789 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:28.698738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:28.699239 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:28.698836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:28.699239 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:28.698923    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:40:28.699239 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:28.698948    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:40:28.699239 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:28.698963    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df94489-vlxg7: secret "image-registry-tls" not found
Jul 10 02:40:28.699239 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:28.698996    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls podName:b15416cc-0665-46f9-af55-f1eb77451bbc nodeName:}" failed. No retries permitted until 2026-07-10 02:40:32.698978983 +0000 UTC m=+42.508244276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls") pod "dns-default-979w2" (UID: "b15416cc-0665-46f9-af55-f1eb77451bbc") : secret "dns-default-metrics-tls" not found
Jul 10 02:40:28.699239 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:28.699019    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls podName:ddb8f374-e401-461b-8618-921fa7187864 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:32.699006054 +0000 UTC m=+42.508271335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls") pod "image-registry-df94489-vlxg7" (UID: "ddb8f374-e401-461b-8618-921fa7187864") : secret "image-registry-tls" not found
Jul 10 02:40:30.011403 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:30.011213    2576 generic.go:356] "Generic (PLEG): container finished" podID="228cc28c-9b29-4a85-a465-040c721653fa" containerID="1b67277364fca51475514982562caf6ca4fdcb17e42730fee761d2b8d6f99aa6" exitCode=0
Jul 10 02:40:30.011864 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:30.011292    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmb92" event={"ID":"228cc28c-9b29-4a85-a465-040c721653fa","Type":"ContainerDied","Data":"1b67277364fca51475514982562caf6ca4fdcb17e42730fee761d2b8d6f99aa6"}
Jul 10 02:40:30.012812 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:30.012790    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fktxn" event={"ID":"b5c7dab0-cc72-4598-8761-2256c99bb318","Type":"ContainerStarted","Data":"2838e280b93036f3f92314eacceeb18230104ca907ac4e81c055f2cbb7152a8a"}
Jul 10 02:40:30.051130 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:30.050686    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fktxn" podStartSLOduration=1.487376 podStartE2EDuration="5.050666644s" totalImagesPullingTime="3.563290644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:30.049667539 +0000 UTC m=+39.858932841" watchObservedRunningTime="2026-07-10 02:40:30.050666644 +0000 UTC m=+39.859931949"
Jul 10 02:40:31.017477 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:31.017444    2576 generic.go:356] "Generic (PLEG): container finished" podID="228cc28c-9b29-4a85-a465-040c721653fa" containerID="82b131dc174070a8a199c3677b474f7878a196472411353e17d26732de370751" exitCode=0
Jul 10 02:40:31.018042 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:31.017530    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmb92" event={"ID":"228cc28c-9b29-4a85-a465-040c721653fa","Type":"ContainerDied","Data":"82b131dc174070a8a199c3677b474f7878a196472411353e17d26732de370751"}
Jul 10 02:40:32.022606 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:32.022566    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmb92" event={"ID":"228cc28c-9b29-4a85-a465-040c721653fa","Type":"ContainerStarted","Data":"a96d68596b81510bf524c8878d2f35e72a9ca0a6a895b74ba54c4a100ac69a10"}
Jul 10 02:40:32.044594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:32.044537    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cmb92" podStartSLOduration=7.104778083 podStartE2EDuration="41.044518663s" totalImagesPullingTime="33.93974058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:32.043173438 +0000 UTC m=+41.852438734" watchObservedRunningTime="2026-07-10 02:40:32.044518663 +0000 UTC m=+41.853783965"
Jul 10 02:40:32.729114 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:32.729076    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:32.729296 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:32.729159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:32.729296 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:32.729241    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:40:32.729296 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:32.729249    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:40:32.729296 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:32.729257    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df94489-vlxg7: secret "image-registry-tls" not found
Jul 10 02:40:32.729422 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:32.729311    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls podName:b15416cc-0665-46f9-af55-f1eb77451bbc nodeName:}" failed. No retries permitted until 2026-07-10 02:40:40.729296226 +0000 UTC m=+50.538561518 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls") pod "dns-default-979w2" (UID: "b15416cc-0665-46f9-af55-f1eb77451bbc") : secret "dns-default-metrics-tls" not found
Jul 10 02:40:32.729422 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:32.729326    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls podName:ddb8f374-e401-461b-8618-921fa7187864 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:40.729320352 +0000 UTC m=+50.538585645 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls") pod "image-registry-df94489-vlxg7" (UID: "ddb8f374-e401-461b-8618-921fa7187864") : secret "image-registry-tls" not found
Jul 10 02:40:33.137965 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.137895    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-lg5nc"]
Jul 10 02:40:33.171419 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.171379    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"]
Jul 10 02:40:33.171585 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.171509    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.174392 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.174369    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 10 02:40:33.174651 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.174627    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 02:40:33.174779 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.174634    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 02:40:33.174779 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.174717    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 10 02:40:33.175670 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.175655    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fhgfm\""
Jul 10 02:40:33.191594 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.191568    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"]
Jul 10 02:40:33.191929 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.191913    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:33.195267 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.195089    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:40:33.195377 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.195318    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 10 02:40:33.195467 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.195447    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7s8h6\""
Jul 10 02:40:33.195549 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.195534    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 10 02:40:33.196758 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.196725    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 10 02:40:33.207457 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.207434    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-lg5nc"]
Jul 10 02:40:33.207562 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.207464    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"]
Jul 10 02:40:33.207562 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.207483    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"]
Jul 10 02:40:33.207562 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.207551    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:33.210208 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.210186    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 10 02:40:33.210318 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.210209    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 02:40:33.210318 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.210246    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 02:40:33.210318 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.210248    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gsw8j\""
Jul 10 02:40:33.211458 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.211443    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 10 02:40:33.232983 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.232958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:33.233113 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.232988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aff6a6-f610-4b8f-93d0-c983270ee97a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.233113 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.233092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2aff6a6-f610-4b8f-93d0-c983270ee97a-tmp\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.233191 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.233116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aff6a6-f610-4b8f-93d0-c983270ee97a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.233191 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.233137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvxt9\" (UniqueName: \"kubernetes.io/projected/e2aff6a6-f610-4b8f-93d0-c983270ee97a-kube-api-access-dvxt9\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.233191 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.233153    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8be658e5-7cb3-4096-b810-3b2791998605-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:33.233293 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.233190    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e2aff6a6-f610-4b8f-93d0-c983270ee97a-snapshots\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.233293 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.233210    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:33.233293 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.233230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljr5b\" (UniqueName: \"kubernetes.io/projected/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-kube-api-access-ljr5b\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:33.233293 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.233246    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2aff6a6-f610-4b8f-93d0-c983270ee97a-serving-cert\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.233293 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.233267    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4ct4\" (UniqueName: \"kubernetes.io/projected/8be658e5-7cb3-4096-b810-3b2791998605-kube-api-access-d4ct4\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:33.242542 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.242519    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"]
Jul 10 02:40:33.255468 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.255444    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-76c954fdf4-rc6fn"]
Jul 10 02:40:33.255578 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.255563    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:33.258431 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.258409    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 02:40:33.258549 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.258415    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-s4xpz\""
Jul 10 02:40:33.258549 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.258441    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 02:40:33.265491 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.265472    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"]
Jul 10 02:40:33.265572 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.265498    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76c954fdf4-rc6fn"]
Jul 10 02:40:33.265606 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.265571    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.267920 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.267901    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 10 02:40:33.268148 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.268129    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 10 02:40:33.268259 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.268238    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 10 02:40:33.268366 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.268353    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 10 02:40:33.268430 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.268353    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 02:40:33.268515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.268498    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xr6l2\""
Jul 10 02:40:33.268621 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.268603    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 02:40:33.334034 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.334197 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e2aff6a6-f610-4b8f-93d0-c983270ee97a-snapshots\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.334197 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334074    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-nginx-conf\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:33.334197 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334095    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aff6a6-f610-4b8f-93d0-c983270ee97a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.334197 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334112    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2aff6a6-f610-4b8f-93d0-c983270ee97a-serving-cert\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.334197 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4ct4\" (UniqueName: \"kubernetes.io/projected/8be658e5-7cb3-4096-b810-3b2791998605-kube-api-access-d4ct4\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:33.334197 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.334412 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:33.334412 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvxt9\" (UniqueName: \"kubernetes.io/projected/e2aff6a6-f610-4b8f-93d0-c983270ee97a-kube-api-access-dvxt9\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.334412 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334367    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aff6a6-f610-4b8f-93d0-c983270ee97a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.334412 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334395    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-default-certificate\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.334613 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:33.334613 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.334477    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 02:40:33.334613 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-stats-auth\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.334613 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.334557    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls podName:df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:33.834536531 +0000 UTC m=+43.643801825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls") pod "cluster-samples-operator-65db858894-n2fbg" (UID: "df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52") : secret "samples-operator-tls" not found
Jul 10 02:40:33.334613 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.334602    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:33.334842 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2aff6a6-f610-4b8f-93d0-c983270ee97a-tmp\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.334842 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.334659    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls podName:8be658e5-7cb3-4096-b810-3b2791998605 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:33.834643443 +0000 UTC m=+43.643908742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-4rvpk" (UID: "8be658e5-7cb3-4096-b810-3b2791998605") : secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:33.334842 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8be658e5-7cb3-4096-b810-3b2791998605-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:33.334842 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334712    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:33.334842 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9k8f\" (UniqueName: \"kubernetes.io/projected/b0ecac30-e48a-4824-b238-ab3453895b35-kube-api-access-c9k8f\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.334842 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.334787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljr5b\" (UniqueName: \"kubernetes.io/projected/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-kube-api-access-ljr5b\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:33.335139 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.335120    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aff6a6-f610-4b8f-93d0-c983270ee97a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.335139 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.335123    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e2aff6a6-f610-4b8f-93d0-c983270ee97a-snapshots\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.335244 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.335193    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aff6a6-f610-4b8f-93d0-c983270ee97a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.335359 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.335340    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8be658e5-7cb3-4096-b810-3b2791998605-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:33.335423 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.335404    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2aff6a6-f610-4b8f-93d0-c983270ee97a-tmp\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.338247 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.338224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2aff6a6-f610-4b8f-93d0-c983270ee97a-serving-cert\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.343254 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.343219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljr5b\" (UniqueName: \"kubernetes.io/projected/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-kube-api-access-ljr5b\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:33.343508 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.343489    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvxt9\" (UniqueName: \"kubernetes.io/projected/e2aff6a6-f610-4b8f-93d0-c983270ee97a-kube-api-access-dvxt9\") pod \"insights-operator-68d9dd94cf-lg5nc\" (UID: \"e2aff6a6-f610-4b8f-93d0-c983270ee97a\") " pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.343573 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.343514    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4ct4\" (UniqueName: \"kubernetes.io/projected/8be658e5-7cb3-4096-b810-3b2791998605-kube-api-access-d4ct4\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:33.435597 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.435504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.435597 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.435549    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-nginx-conf\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:33.435597 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.435573    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.435928 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.435671    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 02:40:33.435928 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.435708    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:33.935687433 +0000 UTC m=+43.744952713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : configmap references non-existent config key: service-ca.crt
Jul 10 02:40:33.435928 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.435733    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:33.935717662 +0000 UTC m=+43.744982946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : secret "router-metrics-certs-default" not found
Jul 10 02:40:33.435928 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.435824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-default-certificate\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.435928 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.435868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-stats-auth\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.435928 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.435913    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:33.436209 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.435937    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9k8f\" (UniqueName: \"kubernetes.io/projected/b0ecac30-e48a-4824-b238-ab3453895b35-kube-api-access-c9k8f\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.436209 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.436042    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:40:33.436209 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.436093    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert podName:3d2951ef-9d07-4f7a-8d18-6e2a0144e216 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:33.936080957 +0000 UTC m=+43.745346261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-ktzsj" (UID: "3d2951ef-9d07-4f7a-8d18-6e2a0144e216") : secret "networking-console-plugin-cert" not found
Jul 10 02:40:33.436319 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.436248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-nginx-conf\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:33.438279 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.438259    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-default-certificate\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.438353 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.438308    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-stats-auth\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.444258 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.444237    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9k8f\" (UniqueName: \"kubernetes.io/projected/b0ecac30-e48a-4824-b238-ab3453895b35-kube-api-access-c9k8f\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.495777 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.495726    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc"
Jul 10 02:40:33.647693 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.647659    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-lg5nc"]
Jul 10 02:40:33.651216 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:40:33.651191    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2aff6a6_f610_4b8f_93d0_c983270ee97a.slice/crio-518fb297ab807485d158a9e82f65ec03e401b1b8171a8044930e8e88617e84be WatchSource:0}: Error finding container 518fb297ab807485d158a9e82f65ec03e401b1b8171a8044930e8e88617e84be: Status 404 returned error can't find the container with id 518fb297ab807485d158a9e82f65ec03e401b1b8171a8044930e8e88617e84be
Jul 10 02:40:33.838822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.838787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:33.838976 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.838870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:33.838976 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.838960    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 02:40:33.839062 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.839006    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:33.839062 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.839025    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls podName:df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:34.839007127 +0000 UTC m=+44.648272427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls") pod "cluster-samples-operator-65db858894-n2fbg" (UID: "df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52") : secret "samples-operator-tls" not found
Jul 10 02:40:33.839062 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.839061    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls podName:8be658e5-7cb3-4096-b810-3b2791998605 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:34.839044541 +0000 UTC m=+44.648309822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-4rvpk" (UID: "8be658e5-7cb3-4096-b810-3b2791998605") : secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:33.939510 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.939469    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.939684 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.939520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:33.939684 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.939632    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:34.93961445 +0000 UTC m=+44.748879744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : configmap references non-existent config key: service-ca.crt
Jul 10 02:40:33.939684 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.939633    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 02:40:33.939879 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.939695    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:34.939686527 +0000 UTC m=+44.748951808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : secret "router-metrics-certs-default" not found
Jul 10 02:40:33.939879 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:33.939722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:33.939879 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.939829    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:40:33.939879 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:33.939860    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert podName:3d2951ef-9d07-4f7a-8d18-6e2a0144e216 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:34.939853266 +0000 UTC m=+44.749118546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-ktzsj" (UID: "3d2951ef-9d07-4f7a-8d18-6e2a0144e216") : secret "networking-console-plugin-cert" not found
Jul 10 02:40:34.027456 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:34.027423    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc" event={"ID":"e2aff6a6-f610-4b8f-93d0-c983270ee97a","Type":"ContainerStarted","Data":"518fb297ab807485d158a9e82f65ec03e401b1b8171a8044930e8e88617e84be"}
Jul 10 02:40:34.848155 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:34.848123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:34.848585 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:34.848192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:34.848585 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:34.848302    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:34.848585 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:34.848306    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 02:40:34.848585 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:34.848370    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls podName:8be658e5-7cb3-4096-b810-3b2791998605 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:36.848351702 +0000 UTC m=+46.657616988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-4rvpk" (UID: "8be658e5-7cb3-4096-b810-3b2791998605") : secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:34.848585 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:34.848388    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls podName:df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:36.848379118 +0000 UTC m=+46.657644399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls") pod "cluster-samples-operator-65db858894-n2fbg" (UID: "df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52") : secret "samples-operator-tls" not found
Jul 10 02:40:34.949441 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:34.949404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:34.949635 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:34.949467    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:34.949635 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:34.949575    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 02:40:34.949761 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:34.949640    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:36.949620472 +0000 UTC m=+46.758885763 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : secret "router-metrics-certs-default" not found
Jul 10 02:40:34.949761 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:34.949668    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:36.949646729 +0000 UTC m=+46.758912022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : configmap references non-existent config key: service-ca.crt
Jul 10 02:40:34.949761 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:34.949704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:34.949896 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:34.949881    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:40:34.949941 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:34.949935    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert podName:3d2951ef-9d07-4f7a-8d18-6e2a0144e216 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:36.949919634 +0000 UTC m=+46.759184927 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-ktzsj" (UID: "3d2951ef-9d07-4f7a-8d18-6e2a0144e216") : secret "networking-console-plugin-cert" not found
Jul 10 02:40:36.866021 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:36.865977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:36.866482 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:36.866119    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:36.866482 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:36.866185    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls podName:8be658e5-7cb3-4096-b810-3b2791998605 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:40.86616954 +0000 UTC m=+50.675434820 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-4rvpk" (UID: "8be658e5-7cb3-4096-b810-3b2791998605") : secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:36.866482 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:36.866205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:36.866482 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:36.866276    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 02:40:36.866482 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:36.866300    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls podName:df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:40.866293192 +0000 UTC m=+50.675558473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls") pod "cluster-samples-operator-65db858894-n2fbg" (UID: "df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52") : secret "samples-operator-tls" not found
Jul 10 02:40:36.966958 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:36.966906    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:36.967114 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:36.967046    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:36.967114 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:36.967082    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:40.967058766 +0000 UTC m=+50.776324059 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : configmap references non-existent config key: service-ca.crt
Jul 10 02:40:36.967225 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:36.967151    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:40:36.967225 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:36.967186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:36.967225 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:36.967196    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert podName:3d2951ef-9d07-4f7a-8d18-6e2a0144e216 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:40.967184028 +0000 UTC m=+50.776449333 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-ktzsj" (UID: "3d2951ef-9d07-4f7a-8d18-6e2a0144e216") : secret "networking-console-plugin-cert" not found
Jul 10 02:40:36.967358 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:36.967246    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 02:40:36.967358 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:36.967276    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:40.967268838 +0000 UTC m=+50.776534119 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : secret "router-metrics-certs-default" not found
Jul 10 02:40:37.036260 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.036225    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc" event={"ID":"e2aff6a6-f610-4b8f-93d0-c983270ee97a","Type":"ContainerStarted","Data":"f1d1d95e2f6b11567c302b09d94f33b18cf411ba4a7c37d18448daf0259f149d"}
Jul 10 02:40:37.052379 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.052324    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc" podStartSLOduration=1.2592369350000001 podStartE2EDuration="4.052306387s" totalImagesPullingTime="2.793069452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:37.052042752 +0000 UTC m=+46.861308066" watchObservedRunningTime="2026-07-10 02:40:37.052306387 +0000 UTC m=+46.861571690"
Jul 10 02:40:37.677867 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.677823    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cfxrv"]
Jul 10 02:40:37.702796 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.702732    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cfxrv"]
Jul 10 02:40:37.702948 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.702849    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.705190 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.705164    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c8l6g\""
Jul 10 02:40:37.705330 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.705243    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 02:40:37.705330 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.705302    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 02:40:37.774462 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.774429    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/239684b3-706b-4ec0-b267-e7f179b0ba03-crio-socket\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.774626 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.774525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.774689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.774638    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.774769 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.774733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/239684b3-706b-4ec0-b267-e7f179b0ba03-data-volume\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.774855 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.774840    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt659\" (UniqueName: \"kubernetes.io/projected/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-api-access-zt659\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.876224 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.876173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/239684b3-706b-4ec0-b267-e7f179b0ba03-crio-socket\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.876689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.876247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.876689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.876318    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.876689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.876358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/239684b3-706b-4ec0-b267-e7f179b0ba03-data-volume\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.876689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.876390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/239684b3-706b-4ec0-b267-e7f179b0ba03-crio-socket\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.876689 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:37.876406    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:40:37.876689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.876428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt659\" (UniqueName: \"kubernetes.io/projected/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-api-access-zt659\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.876689 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:37.876448    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:40:37.876689 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:37.876467    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls podName:239684b3-706b-4ec0-b267-e7f179b0ba03 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:38.376444353 +0000 UTC m=+48.185709633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfxrv" (UID: "239684b3-706b-4ec0-b267-e7f179b0ba03") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:40:37.876689 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:37.876594    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm podName:239684b3-706b-4ec0-b267-e7f179b0ba03 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:38.376581188 +0000 UTC m=+48.185846467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfxrv" (UID: "239684b3-706b-4ec0-b267-e7f179b0ba03") : configmap "kube-rbac-proxy" not found
Jul 10 02:40:37.876689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.876694    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/239684b3-706b-4ec0-b267-e7f179b0ba03-data-volume\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:37.884558 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:37.884537    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt659\" (UniqueName: \"kubernetes.io/projected/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-api-access-zt659\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:38.381048 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:38.381015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:38.381243 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:38.381080    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:38.381243 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:38.381191    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:40:38.381243 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:38.381217    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:40:38.381355 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:38.381258    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls podName:239684b3-706b-4ec0-b267-e7f179b0ba03 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:39.381243469 +0000 UTC m=+49.190508750 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfxrv" (UID: "239684b3-706b-4ec0-b267-e7f179b0ba03") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:40:38.381355 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:38.381276    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm podName:239684b3-706b-4ec0-b267-e7f179b0ba03 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:39.381264355 +0000 UTC m=+49.190529634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfxrv" (UID: "239684b3-706b-4ec0-b267-e7f179b0ba03") : configmap "kube-rbac-proxy" not found
Jul 10 02:40:39.270264 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:39.270234    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cfmzm_22008c3f-7d3c-4151-910c-8950c916a84c/dns-node-resolver/0.log"
Jul 10 02:40:39.390928 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:39.390891    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:39.391087 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:39.391015    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:40:39.391087 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:39.391079    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm podName:239684b3-706b-4ec0-b267-e7f179b0ba03 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:41.391062166 +0000 UTC m=+51.200327446 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfxrv" (UID: "239684b3-706b-4ec0-b267-e7f179b0ba03") : configmap "kube-rbac-proxy" not found
Jul 10 02:40:39.391087 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:39.391016    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:39.391205 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:39.391087    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:40:39.391205 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:39.391201    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls podName:239684b3-706b-4ec0-b267-e7f179b0ba03 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:41.391189815 +0000 UTC m=+51.200455119 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfxrv" (UID: "239684b3-706b-4ec0-b267-e7f179b0ba03") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:40:40.470650 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:40.470623    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r7k5d_01980251-990d-4898-b259-8a05b74de2fc/node-ca/0.log"
Jul 10 02:40:40.804476 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:40.804385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:40.804608 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:40.804477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:40.804608 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:40.804544    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:40:40.804608 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:40.804559    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df94489-vlxg7: secret "image-registry-tls" not found
Jul 10 02:40:40.804608 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:40.804570    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:40:40.804736 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:40.804629    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls podName:b15416cc-0665-46f9-af55-f1eb77451bbc nodeName:}" failed. No retries permitted until 2026-07-10 02:40:56.804614756 +0000 UTC m=+66.613880049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls") pod "dns-default-979w2" (UID: "b15416cc-0665-46f9-af55-f1eb77451bbc") : secret "dns-default-metrics-tls" not found
Jul 10 02:40:40.804736 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:40.804645    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls podName:ddb8f374-e401-461b-8618-921fa7187864 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:56.80463829 +0000 UTC m=+66.613903582 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls") pod "image-registry-df94489-vlxg7" (UID: "ddb8f374-e401-461b-8618-921fa7187864") : secret "image-registry-tls" not found
Jul 10 02:40:40.882279 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:40.882250    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fktxn_b5c7dab0-cc72-4598-8761-2256c99bb318/serve-healthcheck-canary/0.log"
Jul 10 02:40:40.905537 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:40.905505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:40.905654 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:40.905576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:40.905703 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:40.905664    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 02:40:40.905703 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:40.905690    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:40.905807 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:40.905728    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls podName:df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:48.905711191 +0000 UTC m=+58.714976472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls") pod "cluster-samples-operator-65db858894-n2fbg" (UID: "df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52") : secret "samples-operator-tls" not found
Jul 10 02:40:40.905807 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:40.905765    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls podName:8be658e5-7cb3-4096-b810-3b2791998605 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:48.905737485 +0000 UTC m=+58.715002767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-4rvpk" (UID: "8be658e5-7cb3-4096-b810-3b2791998605") : secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:41.006071 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:41.006035    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:41.006222 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:41.006085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:41.006222 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:41.006186    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 02:40:41.006294 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:41.006241    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:49.006227683 +0000 UTC m=+58.815492966 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : secret "router-metrics-certs-default" not found
Jul 10 02:40:41.006334 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:41.006301    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:49.006283156 +0000 UTC m=+58.815548457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : configmap references non-existent config key: service-ca.crt
Jul 10 02:40:41.006334 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:41.006328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:41.006448 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:41.006436    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:40:41.006481 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:41.006471    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert podName:3d2951ef-9d07-4f7a-8d18-6e2a0144e216 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:49.006462784 +0000 UTC m=+58.815728077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-ktzsj" (UID: "3d2951ef-9d07-4f7a-8d18-6e2a0144e216") : secret "networking-console-plugin-cert" not found
Jul 10 02:40:41.409813 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:41.409766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:41.410010 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:41.409848    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:41.410010 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:41.409862    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:40:41.410010 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:41.409935    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls podName:239684b3-706b-4ec0-b267-e7f179b0ba03 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:45.409912825 +0000 UTC m=+55.219178132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfxrv" (UID: "239684b3-706b-4ec0-b267-e7f179b0ba03") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:40:41.410010 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:41.409964    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:40:41.410218 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:41.410018    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm podName:239684b3-706b-4ec0-b267-e7f179b0ba03 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:45.410006401 +0000 UTC m=+55.219271684 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfxrv" (UID: "239684b3-706b-4ec0-b267-e7f179b0ba03") : configmap "kube-rbac-proxy" not found
Jul 10 02:40:45.444214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:45.444162    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:45.444837 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:45.444261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:45.444837 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:45.444323    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:40:45.444837 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:45.444395    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls podName:239684b3-706b-4ec0-b267-e7f179b0ba03 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:53.4443787 +0000 UTC m=+63.253644009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfxrv" (UID: "239684b3-706b-4ec0-b267-e7f179b0ba03") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:40:45.445421 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:45.445400    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/239684b3-706b-4ec0-b267-e7f179b0ba03-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:48.975334 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:48.975293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:40:48.975807 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:48.975389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:40:48.975807 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:48.975460    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 02:40:48.975807 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:48.975513    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:48.975807 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:48.975529    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls podName:df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:04.975513514 +0000 UTC m=+74.784778811 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls") pod "cluster-samples-operator-65db858894-n2fbg" (UID: "df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52") : secret "samples-operator-tls" not found
Jul 10 02:40:48.975807 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:48.975577    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls podName:8be658e5-7cb3-4096-b810-3b2791998605 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:04.975559872 +0000 UTC m=+74.784825167 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-4rvpk" (UID: "8be658e5-7cb3-4096-b810-3b2791998605") : secret "cluster-monitoring-operator-tls" not found
Jul 10 02:40:49.075900 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:49.075868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:40:49.076082 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:49.075942    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:49.076082 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:49.075970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:40:49.076082 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:49.076028    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:40:49.076082 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:49.076071    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 02:40:49.076214 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:49.076098    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:05.076080583 +0000 UTC m=+74.885345876 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : configmap references non-existent config key: service-ca.crt
Jul 10 02:40:49.076214 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:49.076114    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert podName:3d2951ef-9d07-4f7a-8d18-6e2a0144e216 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:05.076107789 +0000 UTC m=+74.885373068 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-ktzsj" (UID: "3d2951ef-9d07-4f7a-8d18-6e2a0144e216") : secret "networking-console-plugin-cert" not found
Jul 10 02:40:49.076214 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:40:49.076124    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs podName:b0ecac30-e48a-4824-b238-ab3453895b35 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:05.076118291 +0000 UTC m=+74.885383571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs") pod "router-default-76c954fdf4-rc6fn" (UID: "b0ecac30-e48a-4824-b238-ab3453895b35") : secret "router-metrics-certs-default" not found
Jul 10 02:40:50.996235 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:50.996203    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ln8cg"
Jul 10 02:40:53.513421 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:53.513364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:53.515636 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:53.515613    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/239684b3-706b-4ec0-b267-e7f179b0ba03-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfxrv\" (UID: \"239684b3-706b-4ec0-b267-e7f179b0ba03\") " pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:53.614393 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:53.614358    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c8l6g\""
Jul 10 02:40:53.622497 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:53.622471    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cfxrv"
Jul 10 02:40:53.744152 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:53.744113    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cfxrv"]
Jul 10 02:40:53.746994 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:40:53.746967    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239684b3_706b_4ec0_b267_e7f179b0ba03.slice/crio-fee095a885201113331694e7df25e49af6fa8693bca7f38f15efa3919c71a091 WatchSource:0}: Error finding container fee095a885201113331694e7df25e49af6fa8693bca7f38f15efa3919c71a091: Status 404 returned error can't find the container with id fee095a885201113331694e7df25e49af6fa8693bca7f38f15efa3919c71a091
Jul 10 02:40:54.077451 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:54.077416    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfxrv" event={"ID":"239684b3-706b-4ec0-b267-e7f179b0ba03","Type":"ContainerStarted","Data":"fee095a885201113331694e7df25e49af6fa8693bca7f38f15efa3919c71a091"}
Jul 10 02:40:56.440764 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.440708    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:56.441225 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.440843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:56.443425 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.443401    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 02:40:56.444255 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.444237    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 02:40:56.453718 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.453688    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d813814-8b23-41d4-852b-ff842e8a5771-original-pull-secret\") pod \"global-pull-secret-syncer-npnft\" (UID: \"3d813814-8b23-41d4-852b-ff842e8a5771\") " pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:56.453876 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.453826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb9fe830-29c0-4665-beea-ffda1ca5ef53-metrics-certs\") pod \"network-metrics-daemon-57zjp\" (UID: \"cb9fe830-29c0-4665-beea-ffda1ca5ef53\") " pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:56.542278 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.542230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77zn\" (UniqueName: \"kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn\") pod \"network-check-target-q4q57\" (UID: \"89160553-786f-443a-8e3b-d6ffb4f4b761\") " pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:56.545056 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.545027    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 02:40:56.555437 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.555406    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 02:40:56.566296 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.566265    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w77zn\" (UniqueName: \"kubernetes.io/projected/89160553-786f-443a-8e3b-d6ffb4f4b761-kube-api-access-w77zn\") pod \"network-check-target-q4q57\" (UID: \"89160553-786f-443a-8e3b-d6ffb4f4b761\") " pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:56.705684 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.705601    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jw76v\""
Jul 10 02:40:56.710800 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.710772    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9pgv\""
Jul 10 02:40:56.713946 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.713919    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57zjp"
Jul 10 02:40:56.714131 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.713970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npnft"
Jul 10 02:40:56.719216 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.719186    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:40:56.844799 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.844765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:56.844998 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.844840    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:56.847343 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.847314    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b15416cc-0665-46f9-af55-f1eb77451bbc-metrics-tls\") pod \"dns-default-979w2\" (UID: \"b15416cc-0665-46f9-af55-f1eb77451bbc\") " pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:56.847515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:56.847494    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") pod \"image-registry-df94489-vlxg7\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") " pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:57.123534 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:57.123502    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w6ngk\""
Jul 10 02:40:57.131559 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:57.131523    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:57.141261 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:57.141236    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8l52l\""
Jul 10 02:40:57.149052 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:57.149030    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-979w2"
Jul 10 02:40:57.534256 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:57.534222    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q4q57"]
Jul 10 02:40:57.537201 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:40:57.537168    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89160553_786f_443a_8e3b_d6ffb4f4b761.slice/crio-2f18799a1813d7f0958628f5d29619c9b12faeeafe3567d863d6e4110a6605fc WatchSource:0}: Error finding container 2f18799a1813d7f0958628f5d29619c9b12faeeafe3567d863d6e4110a6605fc: Status 404 returned error can't find the container with id 2f18799a1813d7f0958628f5d29619c9b12faeeafe3567d863d6e4110a6605fc
Jul 10 02:40:57.762177 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:57.760824    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-57zjp"]
Jul 10 02:40:57.765184 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:57.765156    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-979w2"]
Jul 10 02:40:57.766188 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:57.766163    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-npnft"]
Jul 10 02:40:57.766965 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:57.766931    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-df94489-vlxg7"]
Jul 10 02:40:57.767647 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:40:57.767618    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9fe830_29c0_4665_beea_ffda1ca5ef53.slice/crio-b12e3ee3f86f142559fa3124dfbdc377b4ed4e24c112b41d9967f228b9c48698 WatchSource:0}: Error finding container b12e3ee3f86f142559fa3124dfbdc377b4ed4e24c112b41d9967f228b9c48698: Status 404 returned error can't find the container with id b12e3ee3f86f142559fa3124dfbdc377b4ed4e24c112b41d9967f228b9c48698
Jul 10 02:40:57.768254 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:40:57.768232    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d813814_8b23_41d4_852b_ff842e8a5771.slice/crio-f1184232581e052edc719bc07940c24efd15a6a2e9e009481dc18299439ccc6c WatchSource:0}: Error finding container f1184232581e052edc719bc07940c24efd15a6a2e9e009481dc18299439ccc6c: Status 404 returned error can't find the container with id f1184232581e052edc719bc07940c24efd15a6a2e9e009481dc18299439ccc6c
Jul 10 02:40:57.769429 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:40:57.769406    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15416cc_0665_46f9_af55_f1eb77451bbc.slice/crio-fabe9e7fad1d20d7287adabdc1786e40559439b053d39a96a14fc526adb941f1 WatchSource:0}: Error finding container fabe9e7fad1d20d7287adabdc1786e40559439b053d39a96a14fc526adb941f1: Status 404 returned error can't find the container with id fabe9e7fad1d20d7287adabdc1786e40559439b053d39a96a14fc526adb941f1
Jul 10 02:40:57.770590 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:40:57.770558    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb8f374_e401_461b_8618_921fa7187864.slice/crio-27e44d20a19bc27ba88f4d6127e82ba59e3a72df0e34527e346f425c7e577156 WatchSource:0}: Error finding container 27e44d20a19bc27ba88f4d6127e82ba59e3a72df0e34527e346f425c7e577156: Status 404 returned error can't find the container with id 27e44d20a19bc27ba88f4d6127e82ba59e3a72df0e34527e346f425c7e577156
Jul 10 02:40:58.089303 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:58.089212    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-df94489-vlxg7" event={"ID":"ddb8f374-e401-461b-8618-921fa7187864","Type":"ContainerStarted","Data":"8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696"}
Jul 10 02:40:58.089303 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:58.089269    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-df94489-vlxg7" event={"ID":"ddb8f374-e401-461b-8618-921fa7187864","Type":"ContainerStarted","Data":"27e44d20a19bc27ba88f4d6127e82ba59e3a72df0e34527e346f425c7e577156"}
Jul 10 02:40:58.089527 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:58.089323    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:40:58.090519 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:58.090486    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-979w2" event={"ID":"b15416cc-0665-46f9-af55-f1eb77451bbc","Type":"ContainerStarted","Data":"fabe9e7fad1d20d7287adabdc1786e40559439b053d39a96a14fc526adb941f1"}
Jul 10 02:40:58.091883 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:58.091856    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfxrv" event={"ID":"239684b3-706b-4ec0-b267-e7f179b0ba03","Type":"ContainerStarted","Data":"454b112f6788347930d0df869c2176e2bb5adb8290c1b9f131ee6de2512db85e"}
Jul 10 02:40:58.093214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:58.093192    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-npnft" event={"ID":"3d813814-8b23-41d4-852b-ff842e8a5771","Type":"ContainerStarted","Data":"f1184232581e052edc719bc07940c24efd15a6a2e9e009481dc18299439ccc6c"}
Jul 10 02:40:58.094363 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:58.094334    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q4q57" event={"ID":"89160553-786f-443a-8e3b-d6ffb4f4b761","Type":"ContainerStarted","Data":"2f18799a1813d7f0958628f5d29619c9b12faeeafe3567d863d6e4110a6605fc"}
Jul 10 02:40:58.095473 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:58.095453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-57zjp" event={"ID":"cb9fe830-29c0-4665-beea-ffda1ca5ef53","Type":"ContainerStarted","Data":"b12e3ee3f86f142559fa3124dfbdc377b4ed4e24c112b41d9967f228b9c48698"}
Jul 10 02:40:58.107155 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:40:58.107114    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-df94489-vlxg7" podStartSLOduration=67.107102199 podStartE2EDuration="1m7.107102199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:58.106948843 +0000 UTC m=+67.916214152" watchObservedRunningTime="2026-07-10 02:40:58.107102199 +0000 UTC m=+67.916367501"
Jul 10 02:41:03.113427 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.113392    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfxrv" event={"ID":"239684b3-706b-4ec0-b267-e7f179b0ba03","Type":"ContainerStarted","Data":"b9f47bc1fb2a5fe1373a64ee2d1d08e1df257446940a6a5a2855042fe386cb81"}
Jul 10 02:41:03.114912 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.114881    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-npnft" event={"ID":"3d813814-8b23-41d4-852b-ff842e8a5771","Type":"ContainerStarted","Data":"9e434e2013d0e5318366d9450d6139a38d7e9e14183f3a7fd9d8c2b85b772ad7"}
Jul 10 02:41:03.116307 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.116284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q4q57" event={"ID":"89160553-786f-443a-8e3b-d6ffb4f4b761","Type":"ContainerStarted","Data":"4e66dbecd286f0fadac88c1b4123cf84a8ba0384012d9787265396faaf14497c"}
Jul 10 02:41:03.116422 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.116333    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:41:03.117964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.117937    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-57zjp" event={"ID":"cb9fe830-29c0-4665-beea-ffda1ca5ef53","Type":"ContainerStarted","Data":"cbdf927e783ec18d65230e7bba95acde85cb09f279e63e512a85710389ceb808"}
Jul 10 02:41:03.118080 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.118045    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-57zjp" event={"ID":"cb9fe830-29c0-4665-beea-ffda1ca5ef53","Type":"ContainerStarted","Data":"e0b25fbedf18f328eacac647f8b8b248c5c0db85f3d0b79fd41deb285129bd9c"}
Jul 10 02:41:03.119703 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.119685    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-979w2" event={"ID":"b15416cc-0665-46f9-af55-f1eb77451bbc","Type":"ContainerStarted","Data":"e62e1d73e3670ad07226769e39eb7de80c9202a5315399ed16a02049136125de"}
Jul 10 02:41:03.119819 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.119720    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-979w2" event={"ID":"b15416cc-0665-46f9-af55-f1eb77451bbc","Type":"ContainerStarted","Data":"6dc65ac0f639d04f197c14890b3d8f4446ddc9fe75cd5c42d86cf4b01ccc3ac3"}
Jul 10 02:41:03.119819 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.119797    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-979w2"
Jul 10 02:41:03.129936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.129773    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-npnft" podStartSLOduration=67.194770265 podStartE2EDuration="1m12.129740338s" totalImagesPullingTime="4.934970073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:03.129680223 +0000 UTC m=+72.938945527" watchObservedRunningTime="2026-07-10 02:41:03.129740338 +0000 UTC m=+72.939005641"
Jul 10 02:41:03.145488 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.145427    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q4q57" podStartSLOduration=66.984228136 podStartE2EDuration="1m12.145411775s" totalImagesPullingTime="5.161183639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:03.14433823 +0000 UTC m=+72.953603533" watchObservedRunningTime="2026-07-10 02:41:03.145411775 +0000 UTC m=+72.954677076"
Jul 10 02:41:03.161573 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.161519    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-57zjp" podStartSLOduration=67.249873441 podStartE2EDuration="1m12.161504588s" totalImagesPullingTime="4.911631147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:03.160282392 +0000 UTC m=+72.969547694" watchObservedRunningTime="2026-07-10 02:41:03.161504588 +0000 UTC m=+72.970769890"
Jul 10 02:41:03.176934 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:03.176872    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-979w2" podStartSLOduration=33.266538752 podStartE2EDuration="38.176849128s" totalImagesPullingTime="4.910310376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:03.176301228 +0000 UTC m=+72.985566542" watchObservedRunningTime="2026-07-10 02:41:03.176849128 +0000 UTC m=+72.986114431"
Jul 10 02:41:05.020678 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.020629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:41:05.021177 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.020778    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:41:05.023540 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.023513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8be658e5-7cb3-4096-b810-3b2791998605-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-4rvpk\" (UID: \"8be658e5-7cb3-4096-b810-3b2791998605\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:41:05.023652 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.023539    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-n2fbg\" (UID: \"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:41:05.121651 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.121613    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:41:05.121836 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.121664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:41:05.121836 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.121807    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:41:05.122392 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.122363    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0ecac30-e48a-4824-b238-ab3453895b35-service-ca-bundle\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:41:05.124159 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.124136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ecac30-e48a-4824-b238-ab3453895b35-metrics-certs\") pod \"router-default-76c954fdf4-rc6fn\" (UID: \"b0ecac30-e48a-4824-b238-ab3453895b35\") " pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:41:05.124252 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.124136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d2951ef-9d07-4f7a-8d18-6e2a0144e216-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-ktzsj\" (UID: \"3d2951ef-9d07-4f7a-8d18-6e2a0144e216\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:41:05.305683 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.305592    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7s8h6\""
Jul 10 02:41:05.313561 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.313541    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"
Jul 10 02:41:05.317703 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.317688    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gsw8j\""
Jul 10 02:41:05.325767 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.325729    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"
Jul 10 02:41:05.368327 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.368293    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-s4xpz\""
Jul 10 02:41:05.376775 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.376493    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"
Jul 10 02:41:05.377142 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.377078    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xr6l2\""
Jul 10 02:41:05.385111 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.384970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:41:05.475908 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.475875    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg"]
Jul 10 02:41:05.501777 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.501731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk"]
Jul 10 02:41:05.505437 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:05.505410    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8be658e5_7cb3_4096_b810_3b2791998605.slice/crio-cf1a404e7a61f0ef00736999bca81706b1426f99abd84e9b1144414c68b29b9d WatchSource:0}: Error finding container cf1a404e7a61f0ef00736999bca81706b1426f99abd84e9b1144414c68b29b9d: Status 404 returned error can't find the container with id cf1a404e7a61f0ef00736999bca81706b1426f99abd84e9b1144414c68b29b9d
Jul 10 02:41:05.530606 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.530580    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-ktzsj"]
Jul 10 02:41:05.533858 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:05.533739    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2951ef_9d07_4f7a_8d18_6e2a0144e216.slice/crio-73e164c3f5a88b077fa65ed022b713e14679a7794fd9ea5c3ab3d718faabfe21 WatchSource:0}: Error finding container 73e164c3f5a88b077fa65ed022b713e14679a7794fd9ea5c3ab3d718faabfe21: Status 404 returned error can't find the container with id 73e164c3f5a88b077fa65ed022b713e14679a7794fd9ea5c3ab3d718faabfe21
Jul 10 02:41:05.549944 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:05.549912    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76c954fdf4-rc6fn"]
Jul 10 02:41:05.553535 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:05.553484    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0ecac30_e48a_4824_b238_ab3453895b35.slice/crio-e26b6181436a6bd8f8fd74b14868fe4cf001afa216e0418b4b7fa7d3a957e711 WatchSource:0}: Error finding container e26b6181436a6bd8f8fd74b14868fe4cf001afa216e0418b4b7fa7d3a957e711: Status 404 returned error can't find the container with id e26b6181436a6bd8f8fd74b14868fe4cf001afa216e0418b4b7fa7d3a957e711
Jul 10 02:41:06.135094 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:06.135051    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj" event={"ID":"3d2951ef-9d07-4f7a-8d18-6e2a0144e216","Type":"ContainerStarted","Data":"73e164c3f5a88b077fa65ed022b713e14679a7794fd9ea5c3ab3d718faabfe21"}
Jul 10 02:41:06.136473 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:06.136427    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg" event={"ID":"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52","Type":"ContainerStarted","Data":"7c0983ee544b3f1c459d6c789efa1c487c2c137bca71f116e21b740602900bf2"}
Jul 10 02:41:06.139735 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:06.139675    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfxrv" event={"ID":"239684b3-706b-4ec0-b267-e7f179b0ba03","Type":"ContainerStarted","Data":"e277a3432dfe6918ae71f6ed9b8073cdcf3cd9b343e5d40f96c0c3d206cb7677"}
Jul 10 02:41:06.141522 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:06.141488    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76c954fdf4-rc6fn" event={"ID":"b0ecac30-e48a-4824-b238-ab3453895b35","Type":"ContainerStarted","Data":"b1c54fb6402c20e98938a535f099fe5d3ce64e27c73ab733f36cf45a9b09fb84"}
Jul 10 02:41:06.141636 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:06.141533    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76c954fdf4-rc6fn" event={"ID":"b0ecac30-e48a-4824-b238-ab3453895b35","Type":"ContainerStarted","Data":"e26b6181436a6bd8f8fd74b14868fe4cf001afa216e0418b4b7fa7d3a957e711"}
Jul 10 02:41:06.143667 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:06.143639    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk" event={"ID":"8be658e5-7cb3-4096-b810-3b2791998605","Type":"ContainerStarted","Data":"cf1a404e7a61f0ef00736999bca81706b1426f99abd84e9b1144414c68b29b9d"}
Jul 10 02:41:06.157828 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:06.156829    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cfxrv" podStartSLOduration=17.994265284 podStartE2EDuration="29.156810324s" totalImagesPullingTime="11.16254504s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:40:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:06.155876926 +0000 UTC m=+75.965142232" watchObservedRunningTime="2026-07-10 02:41:06.156810324 +0000 UTC m=+75.966075631"
Jul 10 02:41:06.173665 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:06.173605    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-76c954fdf4-rc6fn" podStartSLOduration=33.17358262 podStartE2EDuration="33.17358262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:06.172040545 +0000 UTC m=+75.981305875" watchObservedRunningTime="2026-07-10 02:41:06.17358262 +0000 UTC m=+75.982847924"
Jul 10 02:41:06.385665 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:06.385574    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:41:06.388811 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:06.388784    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:41:07.149227 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:07.149102    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj" event={"ID":"3d2951ef-9d07-4f7a-8d18-6e2a0144e216","Type":"ContainerStarted","Data":"da57179610bce50dc18a156124264f223678672ffeccb0650be1e2b1653b4216"}
Jul 10 02:41:07.149669 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:07.149601    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:41:07.150758 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:07.150717    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-76c954fdf4-rc6fn"
Jul 10 02:41:07.164514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:07.164462    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-ktzsj" podStartSLOduration=32.962007737 podStartE2EDuration="34.164447135s" totalImagesPullingTime="1.202439398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:07.163701027 +0000 UTC m=+76.972966341" watchObservedRunningTime="2026-07-10 02:41:07.164447135 +0000 UTC m=+76.973712436"
Jul 10 02:41:08.154096 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.154007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg" event={"ID":"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52","Type":"ContainerStarted","Data":"37ee5b65f054d7b0152e3d3ba8b33a5cc3f897bbcabebd57909dffc29135eedc"}
Jul 10 02:41:08.154492 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.154101    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg" event={"ID":"df6b98e4-a66f-4fa5-9a9e-082fe3e6ef52","Type":"ContainerStarted","Data":"262d5cc4c1a4214545db9d0c41c23724f281af8eb169f67d68e7fbcd1f65167e"}
Jul 10 02:41:08.155584 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.155556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk" event={"ID":"8be658e5-7cb3-4096-b810-3b2791998605","Type":"ContainerStarted","Data":"1e0036be3c007c7db2a1f54becd25b1b4f391d321b0d9c84d3798f0825803fb0"}
Jul 10 02:41:08.174397 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.174350    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-n2fbg" podStartSLOduration=32.917006999 podStartE2EDuration="35.174336313s" totalImagesPullingTime="2.257329314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:08.173286287 +0000 UTC m=+77.982551590" watchObservedRunningTime="2026-07-10 02:41:08.174336313 +0000 UTC m=+77.983601614"
Jul 10 02:41:08.191005 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.190950    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-4rvpk" podStartSLOduration=32.916894665 podStartE2EDuration="35.190931275s" totalImagesPullingTime="2.27403661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:08.190415231 +0000 UTC m=+77.999680533" watchObservedRunningTime="2026-07-10 02:41:08.190931275 +0000 UTC m=+78.000196576"
Jul 10 02:41:08.296351 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.296314    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn"]
Jul 10 02:41:08.298491 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.298474    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn"
Jul 10 02:41:08.300864 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.300842    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 10 02:41:08.300967 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.300910    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-595mh\""
Jul 10 02:41:08.310134 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.310106    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn"]
Jul 10 02:41:08.452570 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.452471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a4f5153b-b5f2-4eb5-a79b-a99dcd3e6d26-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-8f5zn\" (UID: \"a4f5153b-b5f2-4eb5-a79b-a99dcd3e6d26\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn"
Jul 10 02:41:08.553677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.553637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a4f5153b-b5f2-4eb5-a79b-a99dcd3e6d26-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-8f5zn\" (UID: \"a4f5153b-b5f2-4eb5-a79b-a99dcd3e6d26\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn"
Jul 10 02:41:08.556077 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.556044    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a4f5153b-b5f2-4eb5-a79b-a99dcd3e6d26-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-8f5zn\" (UID: \"a4f5153b-b5f2-4eb5-a79b-a99dcd3e6d26\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn"
Jul 10 02:41:08.607968 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.607926    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn"
Jul 10 02:41:08.758326 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:08.758295    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn"]
Jul 10 02:41:08.761363 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:08.761339    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4f5153b_b5f2_4eb5_a79b_a99dcd3e6d26.slice/crio-6af44d929d9e2bca14ee1cd3e560f06f79496680e4ea9ed0da492e3c46827416 WatchSource:0}: Error finding container 6af44d929d9e2bca14ee1cd3e560f06f79496680e4ea9ed0da492e3c46827416: Status 404 returned error can't find the container with id 6af44d929d9e2bca14ee1cd3e560f06f79496680e4ea9ed0da492e3c46827416
Jul 10 02:41:09.160339 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:09.160303    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn" event={"ID":"a4f5153b-b5f2-4eb5-a79b-a99dcd3e6d26","Type":"ContainerStarted","Data":"6af44d929d9e2bca14ee1cd3e560f06f79496680e4ea9ed0da492e3c46827416"}
Jul 10 02:41:10.165116 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.165031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn" event={"ID":"a4f5153b-b5f2-4eb5-a79b-a99dcd3e6d26","Type":"ContainerStarted","Data":"769a45a0774066a6fdfdb300664cd2e77dac06bc3341da2583bbffdec9a7846c"}
Jul 10 02:41:10.165545 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.165242    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn"
Jul 10 02:41:10.169795 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.169769    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn"
Jul 10 02:41:10.182007 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.181959    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-8f5zn" podStartSLOduration=1.161595864 podStartE2EDuration="2.181940361s" totalImagesPullingTime="1.020344497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:10.180342708 +0000 UTC m=+79.989608011" watchObservedRunningTime="2026-07-10 02:41:10.181940361 +0000 UTC m=+79.991205664"
Jul 10 02:41:10.371503 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.371468    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-g2w74"]
Jul 10 02:41:10.374740 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.374721    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.377182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.377147    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 10 02:41:10.377182 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.377154    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-v875b\""
Jul 10 02:41:10.377379 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.377157    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 02:41:10.377379 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.377228    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 10 02:41:10.385483 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.385455    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-g2w74"]
Jul 10 02:41:10.567293 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.567247    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e7a9540-db48-4cec-9934-b49772d03eed-metrics-client-ca\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.567293 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.567298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9540-db48-4cec-9934-b49772d03eed-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.567523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.567368    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zrlq\" (UniqueName: \"kubernetes.io/projected/0e7a9540-db48-4cec-9934-b49772d03eed-kube-api-access-2zrlq\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.567523 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.567438    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e7a9540-db48-4cec-9934-b49772d03eed-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.667798 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.667761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e7a9540-db48-4cec-9934-b49772d03eed-metrics-client-ca\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.667798 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.667801    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9540-db48-4cec-9934-b49772d03eed-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.668047 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.667834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zrlq\" (UniqueName: \"kubernetes.io/projected/0e7a9540-db48-4cec-9934-b49772d03eed-kube-api-access-2zrlq\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.668047 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.667867    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e7a9540-db48-4cec-9934-b49772d03eed-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.668527 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.668503    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e7a9540-db48-4cec-9934-b49772d03eed-metrics-client-ca\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.670426 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.670400    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e7a9540-db48-4cec-9934-b49772d03eed-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.670518 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.670463    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9540-db48-4cec-9934-b49772d03eed-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.677046 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.677020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zrlq\" (UniqueName: \"kubernetes.io/projected/0e7a9540-db48-4cec-9934-b49772d03eed-kube-api-access-2zrlq\") pod \"prometheus-operator-64785b6675-g2w74\" (UID: \"0e7a9540-db48-4cec-9934-b49772d03eed\") " pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.683699 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.683675    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74"
Jul 10 02:41:10.815794 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:10.815742    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-g2w74"]
Jul 10 02:41:10.818925 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:10.818870    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7a9540_db48_4cec_9934_b49772d03eed.slice/crio-00669e11a19934e6e9c72509db1a85521530f214ce6501d568eef61cfa5e51a4 WatchSource:0}: Error finding container 00669e11a19934e6e9c72509db1a85521530f214ce6501d568eef61cfa5e51a4: Status 404 returned error can't find the container with id 00669e11a19934e6e9c72509db1a85521530f214ce6501d568eef61cfa5e51a4
Jul 10 02:41:11.169845 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:11.169741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74" event={"ID":"0e7a9540-db48-4cec-9934-b49772d03eed","Type":"ContainerStarted","Data":"00669e11a19934e6e9c72509db1a85521530f214ce6501d568eef61cfa5e51a4"}
Jul 10 02:41:12.175813 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.175783    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74" event={"ID":"0e7a9540-db48-4cec-9934-b49772d03eed","Type":"ContainerStarted","Data":"ed5cee597da94d838be48ebd47fecb49fc9d5ef603af32b9581e21c192673312"}
Jul 10 02:41:12.176170 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.175842    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74" event={"ID":"0e7a9540-db48-4cec-9934-b49772d03eed","Type":"ContainerStarted","Data":"ad19d57e1b7962545e5972ef2d2f784233d6a848ebeb6e4a51e07e3084d8ca6d"}
Jul 10 02:41:12.193826 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.193760    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-g2w74" podStartSLOduration=0.962138907 podStartE2EDuration="2.193722348s" totalImagesPullingTime="1.231583441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:12.191323133 +0000 UTC m=+82.000588436" watchObservedRunningTime="2026-07-10 02:41:12.193722348 +0000 UTC m=+82.002987651"
Jul 10 02:41:12.906649 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.906613    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-794d795678-n48kg"]
Jul 10 02:41:12.910668 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.910650    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:12.913078 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.913049    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 10 02:41:12.913221 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.913181    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 10 02:41:12.913305 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.913289    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 10 02:41:12.913357 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.913308    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 10 02:41:12.914460 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.914445    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 10 02:41:12.914578 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.914564    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 10 02:41:12.914639 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.914590    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 10 02:41:12.914688 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.914655    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sfgqp\""
Jul 10 02:41:12.922580 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.922548    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 10 02:41:12.925382 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.925344    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-794d795678-n48kg"]
Jul 10 02:41:12.984563 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.984516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-service-ca\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:12.984709 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.984581    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-oauth-serving-cert\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:12.984709 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.984632    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-oauth-config\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:12.984709 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.984666    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-serving-cert\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:12.984709 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.984683    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-trusted-ca-bundle\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:12.984709 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.984707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k68d\" (UniqueName: \"kubernetes.io/projected/18bc67ac-8d22-449e-85db-2668bd6ba455-kube-api-access-8k68d\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:12.984907 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:12.984723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-console-config\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.086018 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.085973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-oauth-serving-cert\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.086018 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.086024    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-oauth-config\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.086229 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.086042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-serving-cert\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.086229 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.086057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-trusted-ca-bundle\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.086229 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.086195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k68d\" (UniqueName: \"kubernetes.io/projected/18bc67ac-8d22-449e-85db-2668bd6ba455-kube-api-access-8k68d\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.086342 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.086242    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-console-config\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.086342 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.086286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-service-ca\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.086936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.086910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-oauth-serving-cert\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.087143 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.087119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-service-ca\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.087228 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.087175    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-trusted-ca-bundle\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.087228 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.087174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-console-config\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.088619 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.088598    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-oauth-config\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.088717 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.088700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-serving-cert\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.094390 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.094371    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k68d\" (UniqueName: \"kubernetes.io/projected/18bc67ac-8d22-449e-85db-2668bd6ba455-kube-api-access-8k68d\") pod \"console-794d795678-n48kg\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") " pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.127984 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.127957    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-979w2"
Jul 10 02:41:13.220335 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.220306    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:13.344599 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.344565    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-794d795678-n48kg"]
Jul 10 02:41:13.347658 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:13.347624    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18bc67ac_8d22_449e_85db_2668bd6ba455.slice/crio-fafa414a081fc75a6f1bb8ab277667174e26035f0f380a4028f1fd67f27672b6 WatchSource:0}: Error finding container fafa414a081fc75a6f1bb8ab277667174e26035f0f380a4028f1fd67f27672b6: Status 404 returned error can't find the container with id fafa414a081fc75a6f1bb8ab277667174e26035f0f380a4028f1fd67f27672b6
Jul 10 02:41:13.759210 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.759178    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"]
Jul 10 02:41:13.763546 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.763525    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.765854 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.765834    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 10 02:41:13.765968 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.765895    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5zx7l\""
Jul 10 02:41:13.766120 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.766108    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 10 02:41:13.766867 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.766854    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 10 02:41:13.774809 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.774787    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-k4hqs"]
Jul 10 02:41:13.777835 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.777814    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"]
Jul 10 02:41:13.777932 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.777918    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.780096 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.780074    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dlwvz\""
Jul 10 02:41:13.780230 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.780212    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 02:41:13.780303 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.780263    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 02:41:13.780303 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.780288    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 02:41:13.791885 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.791862    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-accelerators-collector-config\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.791899    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d8f513ce-213d-41e2-8d62-c6b811c0afce-root\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.791924    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/937ec168-89ef-4178-aeb2-c2e1c6c910b2-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.791950    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8f513ce-213d-41e2-8d62-c6b811c0afce-metrics-client-ca\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792062    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6ctr\" (UniqueName: \"kubernetes.io/projected/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-api-access-x6ctr\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/937ec168-89ef-4178-aeb2-c2e1c6c910b2-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792275    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f513ce-213d-41e2-8d62-c6b811c0afce-sys\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792304    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b67p6\" (UniqueName: \"kubernetes.io/projected/d8f513ce-213d-41e2-8d62-c6b811c0afce-kube-api-access-b67p6\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-textfile\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-tls\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-wtmp\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.792624 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.792528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893354 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.893354 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.893596 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.893596 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893406    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f513ce-213d-41e2-8d62-c6b811c0afce-sys\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893596 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893441    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b67p6\" (UniqueName: \"kubernetes.io/projected/d8f513ce-213d-41e2-8d62-c6b811c0afce-kube-api-access-b67p6\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893596 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:41:13.893511    2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 10 02:41:13.893596 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893509    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f513ce-213d-41e2-8d62-c6b811c0afce-sys\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893596 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:41:13.893584    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-tls podName:937ec168-89ef-4178-aeb2-c2e1c6c910b2 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:14.393562883 +0000 UTC m=+84.202828186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-g2zrn" (UID: "937ec168-89ef-4178-aeb2-c2e1c6c910b2") : secret "kube-state-metrics-tls" not found
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-textfile\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-tls\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-wtmp\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893719    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-accelerators-collector-config\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d8f513ce-213d-41e2-8d62-c6b811c0afce-root\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:41:13.893817    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893849    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/937ec168-89ef-4178-aeb2-c2e1c6c910b2-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:41:13.893882    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-tls podName:d8f513ce-213d-41e2-8d62-c6b811c0afce nodeName:}" failed. No retries permitted until 2026-07-10 02:41:14.393865339 +0000 UTC m=+84.203130625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-tls") pod "node-exporter-k4hqs" (UID: "d8f513ce-213d-41e2-8d62-c6b811c0afce") : secret "node-exporter-tls" not found
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8f513ce-213d-41e2-8d62-c6b811c0afce-metrics-client-ca\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.893936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893937    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6ctr\" (UniqueName: \"kubernetes.io/projected/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-api-access-x6ctr\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.894460 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.893969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/937ec168-89ef-4178-aeb2-c2e1c6c910b2-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.894460 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.894029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d8f513ce-213d-41e2-8d62-c6b811c0afce-root\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.894460 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.894158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-wtmp\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.894460 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.894345    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-textfile\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.894460 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.894375    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/937ec168-89ef-4178-aeb2-c2e1c6c910b2-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.894710 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.894564    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-accelerators-collector-config\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.894710 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.894587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.894710 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.894640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/937ec168-89ef-4178-aeb2-c2e1c6c910b2-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.894710 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.894676    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8f513ce-213d-41e2-8d62-c6b811c0afce-metrics-client-ca\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.896496 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.896472    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.896781 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.896739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:13.902044 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.902020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b67p6\" (UniqueName: \"kubernetes.io/projected/d8f513ce-213d-41e2-8d62-c6b811c0afce-kube-api-access-b67p6\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:13.902287 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:13.902267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6ctr\" (UniqueName: \"kubernetes.io/projected/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-api-access-x6ctr\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:14.184491 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.184399    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-794d795678-n48kg" event={"ID":"18bc67ac-8d22-449e-85db-2668bd6ba455","Type":"ContainerStarted","Data":"fafa414a081fc75a6f1bb8ab277667174e26035f0f380a4028f1fd67f27672b6"}
Jul 10 02:41:14.398128 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.398091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:14.398520 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.398144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-tls\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:14.400856 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.400829    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8f513ce-213d-41e2-8d62-c6b811c0afce-node-exporter-tls\") pod \"node-exporter-k4hqs\" (UID: \"d8f513ce-213d-41e2-8d62-c6b811c0afce\") " pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:14.400996 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.400976    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/937ec168-89ef-4178-aeb2-c2e1c6c910b2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-g2zrn\" (UID: \"937ec168-89ef-4178-aeb2-c2e1c6c910b2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:14.672470 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.672433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"
Jul 10 02:41:14.689854 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.689138    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k4hqs"
Jul 10 02:41:14.704961 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:14.704913    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f513ce_213d_41e2_8d62_c6b811c0afce.slice/crio-8cc9bfbf24993afb8abcbc8e48491a5091d796de89889b7c3599de6ffd989dc0 WatchSource:0}: Error finding container 8cc9bfbf24993afb8abcbc8e48491a5091d796de89889b7c3599de6ffd989dc0: Status 404 returned error can't find the container with id 8cc9bfbf24993afb8abcbc8e48491a5091d796de89889b7c3599de6ffd989dc0
Jul 10 02:41:14.823742 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.823712    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:41:14.829087 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.829054    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.837088 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.837063    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 02:41:14.837217 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.837161    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 02:41:14.837796 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.837772    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 02:41:14.841644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.841278    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 02:41:14.841644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.841296    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 02:41:14.841644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.841315    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tx2fg\""
Jul 10 02:41:14.841644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.841486    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 02:41:14.841644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.841562    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 02:41:14.849545 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.849519    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 02:41:14.850016 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.849992    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:41:14.851104 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.851084    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 02:41:14.856218 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.856196    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-g2zrn"]
Jul 10 02:41:14.860716 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:14.860687    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod937ec168_89ef_4178_aeb2_c2e1c6c910b2.slice/crio-50d8893833eedf2c79157379f77969330b921965c6b1fc7e4136f5f30b5c83db WatchSource:0}: Error finding container 50d8893833eedf2c79157379f77969330b921965c6b1fc7e4136f5f30b5c83db: Status 404 returned error can't find the container with id 50d8893833eedf2c79157379f77969330b921965c6b1fc7e4136f5f30b5c83db
Jul 10 02:41:14.902395 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902355    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902558 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902558 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902558 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902558 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902512    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7pfv\" (UniqueName: \"kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-kube-api-access-b7pfv\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902777 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902574    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902777 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902777 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902777 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902777 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902989 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-out\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902989 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:14.902989 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:14.902887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-web-config\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.003602 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.003562    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.003819 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.003616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.003819 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.003644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.003941 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.003818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-out\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.003941 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.003880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.003941 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.003907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-web-config\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.003941 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.003930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.004140 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.003983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.004140 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.004007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.004140 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.004033    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.004140 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.004065    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7pfv\" (UniqueName: \"kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-kube-api-access-b7pfv\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.004140 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.004116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.004366 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.004183    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.004582 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.004557    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.004963 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:41:15.004918    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-trusted-ca-bundle podName:4d16a64b-256e-4780-8e59-9ae1c636f8c9 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:15.504891856 +0000 UTC m=+85.314157156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9") : configmap references non-existent config key: ca-bundle.crt
Jul 10 02:41:15.006047 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.005690    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.007183 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.007090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.007183 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.007099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.007442 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.007384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.007680 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.007656    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-out\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.007924 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.007873    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.008651 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.008627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.009412 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.009369    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-web-config\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.009628 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.009607    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.010147 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.010117    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.016442 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.016423    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7pfv\" (UniqueName: \"kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-kube-api-access-b7pfv\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.190207 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.190164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4hqs" event={"ID":"d8f513ce-213d-41e2-8d62-c6b811c0afce","Type":"ContainerStarted","Data":"8cc9bfbf24993afb8abcbc8e48491a5091d796de89889b7c3599de6ffd989dc0"}
Jul 10 02:41:15.191493 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.191464    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn" event={"ID":"937ec168-89ef-4178-aeb2-c2e1c6c910b2","Type":"ContainerStarted","Data":"50d8893833eedf2c79157379f77969330b921965c6b1fc7e4136f5f30b5c83db"}
Jul 10 02:41:15.508990 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.508942    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.509940 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.509909    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:15.741172 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:15.741128    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:41:16.198085 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:16.198053    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:41:16.198729 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:16.198700    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-794d795678-n48kg" event={"ID":"18bc67ac-8d22-449e-85db-2668bd6ba455","Type":"ContainerStarted","Data":"6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256"}
Jul 10 02:41:16.199693 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:16.199661    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d16a64b_256e_4780_8e59_9ae1c636f8c9.slice/crio-57cc4a64816ba2c75fb6a9cf2da83759f68405dd94f0872bdee39c502c4c83be WatchSource:0}: Error finding container 57cc4a64816ba2c75fb6a9cf2da83759f68405dd94f0872bdee39c502c4c83be: Status 404 returned error can't find the container with id 57cc4a64816ba2c75fb6a9cf2da83759f68405dd94f0872bdee39c502c4c83be
Jul 10 02:41:16.219158 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:16.218566    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-794d795678-n48kg" podStartSLOduration=1.470748 podStartE2EDuration="4.218543949s" totalImagesPullingTime="2.747795949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:16.217967228 +0000 UTC m=+86.027232531" watchObservedRunningTime="2026-07-10 02:41:16.218543949 +0000 UTC m=+86.027809256"
Jul 10 02:41:17.136771 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.136715    2576 patch_prober.go:28] interesting pod/image-registry-df94489-vlxg7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 02:41:17.137231 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.136811    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-df94489-vlxg7" podUID="ddb8f374-e401-461b-8618-921fa7187864" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:41:17.204513 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.204480    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerStarted","Data":"57cc4a64816ba2c75fb6a9cf2da83759f68405dd94f0872bdee39c502c4c83be"}
Jul 10 02:41:17.729907 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.729878    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-657c76b9bf-mw97d"]
Jul 10 02:41:17.734721 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.734694    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.737441 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.737273    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 10 02:41:17.737441 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.737287    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 10 02:41:17.737441 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.737292    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9ff2t\""
Jul 10 02:41:17.737441 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.737273    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 10 02:41:17.737441 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.737450    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 10 02:41:17.737772 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.737532    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 10 02:41:17.738373 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.738355    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c99vc0ng2of3o\""
Jul 10 02:41:17.745579 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.745557    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-657c76b9bf-mw97d"]
Jul 10 02:41:17.829294 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.829251    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.829294 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.829294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.829506 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.829320    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-tls\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.829506 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.829434    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8770b5b7-4639-4784-926c-f4fccb16d00c-metrics-client-ca\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.829573 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.829560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8zff\" (UniqueName: \"kubernetes.io/projected/8770b5b7-4639-4784-926c-f4fccb16d00c-kube-api-access-h8zff\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.829605 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.829591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-grpc-tls\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.829659 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.829632    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.829701 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.829665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.930247 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.930192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-tls\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.930437 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.930260    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8770b5b7-4639-4784-926c-f4fccb16d00c-metrics-client-ca\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.930437 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.930336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8zff\" (UniqueName: \"kubernetes.io/projected/8770b5b7-4639-4784-926c-f4fccb16d00c-kube-api-access-h8zff\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.930437 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.930362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-grpc-tls\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.930620 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.930522    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.930620 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.930569    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.930721 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.930630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.930721 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.930664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.931202 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.931176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8770b5b7-4639-4784-926c-f4fccb16d00c-metrics-client-ca\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.933445 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.933409    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.933547 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.933415    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.933547 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.933447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-grpc-tls\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.933547 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.933483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-tls\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.933547 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.933524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.933826 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.933809    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8770b5b7-4639-4784-926c-f4fccb16d00c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:17.937842 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:17.937821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8zff\" (UniqueName: \"kubernetes.io/projected/8770b5b7-4639-4784-926c-f4fccb16d00c-kube-api-access-h8zff\") pod \"thanos-querier-657c76b9bf-mw97d\" (UID: \"8770b5b7-4639-4784-926c-f4fccb16d00c\") " pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:18.046106 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.046067    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:18.187413 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.187385    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-657c76b9bf-mw97d"]
Jul 10 02:41:18.189543 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:18.189518    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8770b5b7_4639_4784_926c_f4fccb16d00c.slice/crio-4e714c3d25b8bbdd2e9e2a873c4d59f6ad3d7dc988bb5c89ce1f1d2c344e15fd WatchSource:0}: Error finding container 4e714c3d25b8bbdd2e9e2a873c4d59f6ad3d7dc988bb5c89ce1f1d2c344e15fd: Status 404 returned error can't find the container with id 4e714c3d25b8bbdd2e9e2a873c4d59f6ad3d7dc988bb5c89ce1f1d2c344e15fd
Jul 10 02:41:18.210592 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.210557    2576 generic.go:356] "Generic (PLEG): container finished" podID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerID="084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b" exitCode=0
Jul 10 02:41:18.210721 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.210607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerDied","Data":"084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b"}
Jul 10 02:41:18.212992 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.212967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn" event={"ID":"937ec168-89ef-4178-aeb2-c2e1c6c910b2","Type":"ContainerStarted","Data":"73e42cc574cc860736faf22282727ae2ee2a8ecbac4d13713b2f7a02f6cacb11"}
Jul 10 02:41:18.213085 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.213025    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn" event={"ID":"937ec168-89ef-4178-aeb2-c2e1c6c910b2","Type":"ContainerStarted","Data":"04effc098d2e23311a86dd3d3fa3a0b1d8b11faf154552f2bdd023dd377d1d15"}
Jul 10 02:41:18.213085 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.213047    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn" event={"ID":"937ec168-89ef-4178-aeb2-c2e1c6c910b2","Type":"ContainerStarted","Data":"013eefc75301949e8a0da68bcd09599222d68f4304b2eff10a2071e2561bdb18"}
Jul 10 02:41:18.214814 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.214794    2576 generic.go:356] "Generic (PLEG): container finished" podID="d8f513ce-213d-41e2-8d62-c6b811c0afce" containerID="c870046921346664a8f0fab905a8335d9c96175d0d8340b397c8f51320d173be" exitCode=0
Jul 10 02:41:18.214971 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.214822    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4hqs" event={"ID":"d8f513ce-213d-41e2-8d62-c6b811c0afce","Type":"ContainerDied","Data":"c870046921346664a8f0fab905a8335d9c96175d0d8340b397c8f51320d173be"}
Jul 10 02:41:18.216272 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.216242    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d" event={"ID":"8770b5b7-4639-4784-926c-f4fccb16d00c","Type":"ContainerStarted","Data":"4e714c3d25b8bbdd2e9e2a873c4d59f6ad3d7dc988bb5c89ce1f1d2c344e15fd"}
Jul 10 02:41:18.254427 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:18.254350    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-g2zrn" podStartSLOduration=2.634884329 podStartE2EDuration="5.254331654s" totalImagesPullingTime="2.619447325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:18.253612953 +0000 UTC m=+88.062878256" watchObservedRunningTime="2026-07-10 02:41:18.254331654 +0000 UTC m=+88.063596956"
Jul 10 02:41:19.104173 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.104146    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:41:19.224845 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.224792    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4hqs" event={"ID":"d8f513ce-213d-41e2-8d62-c6b811c0afce","Type":"ContainerStarted","Data":"3e972aab70d9f217c72d7a4b60d09a1c2bf95cd354018931e58e09c91049780d"}
Jul 10 02:41:19.225218 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.224865    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4hqs" event={"ID":"d8f513ce-213d-41e2-8d62-c6b811c0afce","Type":"ContainerStarted","Data":"9abf2c6449a2776440317ed69e4cf1fb9fb97412ad8b343a0e44a7edf3f91fd0"}
Jul 10 02:41:19.242893 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.242829    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-k4hqs" podStartSLOduration=3.385671841 podStartE2EDuration="6.242809063s" totalImagesPullingTime="2.775224647s" totalInitContainerRuntime="81.912575ms" isStatefulPod=true podCreationTimestamp="2026-07-10 02:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:19.242578365 +0000 UTC m=+89.051843666" watchObservedRunningTime="2026-07-10 02:41:19.242809063 +0000 UTC m=+89.052074366"
Jul 10 02:41:19.979212 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.978803    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:41:19.983556 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.983527    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:19.986858 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.986600    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 02:41:19.986858 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.986623    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 02:41:19.986858 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.986785    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 02:41:19.987209 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.986954    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 02:41:19.987209 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.987123    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 02:41:19.987370 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.987241    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 02:41:19.987487 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.987468    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 02:41:19.987901 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.987807    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 02:41:19.988006 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.987970    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1j680n36t9m04\""
Jul 10 02:41:19.988073 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.988013    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 02:41:19.988127 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.988069    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 02:41:19.988127 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.988018    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 02:41:19.988224 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.988198    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 02:41:19.988277 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.988228    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 02:41:19.988323 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.988293    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 02:41:19.988358 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.988292    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2kq2j\""
Jul 10 02:41:19.991220 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:19.991036    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 02:41:20.001075 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.001042    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:41:20.057171 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-web-config\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057171 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057400 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057220    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-476vv\" (UniqueName: \"kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-kube-api-access-476vv\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057400 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057400 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057400 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057367    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057400 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057397    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057588 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057425    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057588 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057588 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057588 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057523    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057588 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057603    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057638    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057692    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.057822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.058007 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057821    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.058007 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057855    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.058007 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057885    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config-out\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.058007 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.057962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159309 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159256    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159488 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159315    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159488 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159488 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159488 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159488 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159438    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159488 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159552    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config-out\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159619    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-web-config\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-476vv\" (UniqueName: \"kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-kube-api-access-476vv\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.159822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.160294 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159846    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.160294 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.160294 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.159900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.160294 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.160133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.160492 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.160399    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.160655 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.160629    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.162423 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.162391    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.163329 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.162988    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.167471 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.164246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.167471 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.166804    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.167471 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.167016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.167471 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.167172    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.168121 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.168091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.171964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.168802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.171964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.170586    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.171964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.170627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.171964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.171366    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-476vv\" (UniqueName: \"kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-kube-api-access-476vv\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.171964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.171386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.171964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.171461    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.171964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.171899    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.172357 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.172211    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config-out\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.172522 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.172483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-web-config\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.173033 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.172987    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.295850 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.295768    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:20.814201 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:20.814175    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:41:20.819164 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:41:20.819106    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de78ec6_0c81_444a_b03c_c5c508e6e48b.slice/crio-be127db0493cdb6ca394384fa8e8e680550436960b3781b072d01486b1df067f WatchSource:0}: Error finding container be127db0493cdb6ca394384fa8e8e680550436960b3781b072d01486b1df067f: Status 404 returned error can't find the container with id be127db0493cdb6ca394384fa8e8e680550436960b3781b072d01486b1df067f
Jul 10 02:41:21.236981 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.236946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d" event={"ID":"8770b5b7-4639-4784-926c-f4fccb16d00c","Type":"ContainerStarted","Data":"a8067f70cb4308931129ca928873af53b95227abc14597d093c495636dc0ad60"}
Jul 10 02:41:21.237186 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.237023    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d" event={"ID":"8770b5b7-4639-4784-926c-f4fccb16d00c","Type":"ContainerStarted","Data":"a39273c36d95df6fa391b31698da310e72f28bb421c3c3990b694d6062c796f0"}
Jul 10 02:41:21.237186 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.237111    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d" event={"ID":"8770b5b7-4639-4784-926c-f4fccb16d00c","Type":"ContainerStarted","Data":"9c476081f2c7c7a8f48f61baf406302be5218d5f14be80f736d7f742cea77cf4"}
Jul 10 02:41:21.238646 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.238618    2576 generic.go:356] "Generic (PLEG): container finished" podID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerID="a057d8daea96344403dcf8c458d8286c87f34337726f7fdd888f48f0d8dc8af4" exitCode=0
Jul 10 02:41:21.238769 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.238706    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerDied","Data":"a057d8daea96344403dcf8c458d8286c87f34337726f7fdd888f48f0d8dc8af4"}
Jul 10 02:41:21.238842 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.238796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerStarted","Data":"be127db0493cdb6ca394384fa8e8e680550436960b3781b072d01486b1df067f"}
Jul 10 02:41:21.242884 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.242861    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerStarted","Data":"eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b"}
Jul 10 02:41:21.243006 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.242922    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerStarted","Data":"be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2"}
Jul 10 02:41:21.243006 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.242950    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerStarted","Data":"2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb"}
Jul 10 02:41:21.243006 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.242973    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerStarted","Data":"c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66"}
Jul 10 02:41:21.243006 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:21.243003    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerStarted","Data":"e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730"}
Jul 10 02:41:22.251675 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:22.251636    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerStarted","Data":"bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068"}
Jul 10 02:41:22.255075 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:22.255041    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d" event={"ID":"8770b5b7-4639-4784-926c-f4fccb16d00c","Type":"ContainerStarted","Data":"0a551a531727b53f286e2838ad2f28fbf93d0ebcaf55f33d78cb19d7df8ce2d3"}
Jul 10 02:41:22.255204 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:22.255116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d" event={"ID":"8770b5b7-4639-4784-926c-f4fccb16d00c","Type":"ContainerStarted","Data":"f5a2099b37c8e516e8bf98e03b1fda4b5df8b9b4acc0c0754f1ee2b9ab56dce0"}
Jul 10 02:41:22.255204 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:22.255148    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d" event={"ID":"8770b5b7-4639-4784-926c-f4fccb16d00c","Type":"ContainerStarted","Data":"d234d578fc161e0f5d56de1c643fe4f1f610d5de22a18504a553821945605406"}
Jul 10 02:41:22.255204 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:22.255183    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:22.283711 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:22.283648    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.938931991 podStartE2EDuration="8.283632382s" totalImagesPullingTime="4.321470841s" totalInitContainerRuntime="23.22955ms" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:22.282036596 +0000 UTC m=+92.091301899" watchObservedRunningTime="2026-07-10 02:41:22.283632382 +0000 UTC m=+92.092897683"
Jul 10 02:41:22.312389 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:22.311932    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d" podStartSLOduration=2.076255192 podStartE2EDuration="5.311908349s" totalImagesPullingTime="3.235653157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:22.305741646 +0000 UTC m=+92.115006951" watchObservedRunningTime="2026-07-10 02:41:22.311908349 +0000 UTC m=+92.121173654"
Jul 10 02:41:23.220434 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:23.220391    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:23.220641 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:23.220470    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:23.226306 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:23.226281    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:23.262892 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:23.262862    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:25.281028 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:25.280996    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerStarted","Data":"4246ba564a8fd60286aced061e9eee8a1f013683755903bced414043786b0108"}
Jul 10 02:41:25.281455 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:25.281087    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerStarted","Data":"a4d19492e0ced72800fa8811e07eff6ce4a61cf40bf1fcdcb78ff4ce4f880b0b"}
Jul 10 02:41:25.281455 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:25.281113    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerStarted","Data":"53d5efb46305e33b33ee9ff07b63a986ee82b9f7c8bac20248def31c30c7e305"}
Jul 10 02:41:25.281455 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:25.281137    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerStarted","Data":"a0ea2d8993ebf236d4522c17f03f77cb02a43cd31a82b7ed293f9ede6a801700"}
Jul 10 02:41:25.281455 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:25.281159    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerStarted","Data":"bf68c2c7c8127af53e6177f3fd9970c9b3ae9c39e5044dc37af882c418f736a0"}
Jul 10 02:41:26.290215 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:26.290179    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerStarted","Data":"044dde645f38752db78d2825136a45309ff08383d3f016d665827175d4f6bf16"}
Jul 10 02:41:26.324892 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:26.324827    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.689074407 podStartE2EDuration="7.32480257s" totalImagesPullingTime="3.609636062s" totalInitContainerRuntime="26.092101ms" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:26.322919474 +0000 UTC m=+96.132184787" watchObservedRunningTime="2026-07-10 02:41:26.32480257 +0000 UTC m=+96.134067874"
Jul 10 02:41:28.267274 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:28.267246    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-657c76b9bf-mw97d"
Jul 10 02:41:28.535568 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:28.535479    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-df94489-vlxg7"]
Jul 10 02:41:29.649290 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:29.649251    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-794d795678-n48kg"]
Jul 10 02:41:30.296963 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:30.296909    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:41:34.128334 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:34.128296    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q4q57"
Jul 10 02:41:52.407008 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:52.406971    2576 generic.go:356] "Generic (PLEG): container finished" podID="e2aff6a6-f610-4b8f-93d0-c983270ee97a" containerID="f1d1d95e2f6b11567c302b09d94f33b18cf411ba4a7c37d18448daf0259f149d" exitCode=0
Jul 10 02:41:52.407471 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:52.407047    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc" event={"ID":"e2aff6a6-f610-4b8f-93d0-c983270ee97a","Type":"ContainerDied","Data":"f1d1d95e2f6b11567c302b09d94f33b18cf411ba4a7c37d18448daf0259f149d"}
Jul 10 02:41:52.407541 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:52.407522    2576 scope.go:117] "RemoveContainer" containerID="f1d1d95e2f6b11567c302b09d94f33b18cf411ba4a7c37d18448daf0259f149d"
Jul 10 02:41:53.413283 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.413250    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lg5nc" event={"ID":"e2aff6a6-f610-4b8f-93d0-c983270ee97a","Type":"ContainerStarted","Data":"143d3291a422cae467d4005cdf4e0cab1c895f1640ae51ee5a72cc6fed59dc34"}
Jul 10 02:41:53.563405 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.563369    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-df94489-vlxg7" podUID="ddb8f374-e401-461b-8618-921fa7187864" containerName="registry" containerID="cri-o://8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696" gracePeriod=30
Jul 10 02:41:53.810165 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.810141    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:41:53.972278 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.972250    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-trusted-ca\") pod \"ddb8f374-e401-461b-8618-921fa7187864\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") "
Jul 10 02:41:53.972453 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.972298    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-bound-sa-token\") pod \"ddb8f374-e401-461b-8618-921fa7187864\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") "
Jul 10 02:41:53.972453 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.972341    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-image-registry-private-configuration\") pod \"ddb8f374-e401-461b-8618-921fa7187864\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") "
Jul 10 02:41:53.972453 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.972386    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ddb8f374-e401-461b-8618-921fa7187864-ca-trust-extracted\") pod \"ddb8f374-e401-461b-8618-921fa7187864\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") "
Jul 10 02:41:53.972453 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.972414    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-installation-pull-secrets\") pod \"ddb8f374-e401-461b-8618-921fa7187864\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") "
Jul 10 02:41:53.972670 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.972550    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") pod \"ddb8f374-e401-461b-8618-921fa7187864\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") "
Jul 10 02:41:53.972670 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.972630    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-registry-certificates\") pod \"ddb8f374-e401-461b-8618-921fa7187864\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") "
Jul 10 02:41:53.972670 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.972649    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzrb4\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-kube-api-access-fzrb4\") pod \"ddb8f374-e401-461b-8618-921fa7187864\" (UID: \"ddb8f374-e401-461b-8618-921fa7187864\") "
Jul 10 02:41:53.972855 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.972799    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ddb8f374-e401-461b-8618-921fa7187864" (UID: "ddb8f374-e401-461b-8618-921fa7187864"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:41:53.973214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.973136    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ddb8f374-e401-461b-8618-921fa7187864" (UID: "ddb8f374-e401-461b-8618-921fa7187864"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:41:53.973214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.973152    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-trusted-ca\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:53.975460 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.975407    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ddb8f374-e401-461b-8618-921fa7187864" (UID: "ddb8f374-e401-461b-8618-921fa7187864"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:41:53.975460 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.975434    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-kube-api-access-fzrb4" (OuterVolumeSpecName: "kube-api-access-fzrb4") pod "ddb8f374-e401-461b-8618-921fa7187864" (UID: "ddb8f374-e401-461b-8618-921fa7187864"). InnerVolumeSpecName "kube-api-access-fzrb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:41:53.975631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.975495    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ddb8f374-e401-461b-8618-921fa7187864" (UID: "ddb8f374-e401-461b-8618-921fa7187864"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:41:53.975773 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.975725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ddb8f374-e401-461b-8618-921fa7187864" (UID: "ddb8f374-e401-461b-8618-921fa7187864"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:41:53.975773 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.975763    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ddb8f374-e401-461b-8618-921fa7187864" (UID: "ddb8f374-e401-461b-8618-921fa7187864"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:41:53.981623 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:53.981597    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddb8f374-e401-461b-8618-921fa7187864-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ddb8f374-e401-461b-8618-921fa7187864" (UID: "ddb8f374-e401-461b-8618-921fa7187864"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:41:54.074418 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.074379    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-bound-sa-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:54.074418 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.074418    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-image-registry-private-configuration\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:54.074418 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.074428    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ddb8f374-e401-461b-8618-921fa7187864-ca-trust-extracted\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:54.074668 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.074438    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ddb8f374-e401-461b-8618-921fa7187864-installation-pull-secrets\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:54.074668 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.074447    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-registry-tls\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:54.074668 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.074455    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ddb8f374-e401-461b-8618-921fa7187864-registry-certificates\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:54.074668 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.074463    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzrb4\" (UniqueName: \"kubernetes.io/projected/ddb8f374-e401-461b-8618-921fa7187864-kube-api-access-fzrb4\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:54.418531 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.418442    2576 generic.go:356] "Generic (PLEG): container finished" podID="ddb8f374-e401-461b-8618-921fa7187864" containerID="8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696" exitCode=0
Jul 10 02:41:54.418531 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.418487    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-df94489-vlxg7"
Jul 10 02:41:54.418531 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.418505    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-df94489-vlxg7" event={"ID":"ddb8f374-e401-461b-8618-921fa7187864","Type":"ContainerDied","Data":"8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696"}
Jul 10 02:41:54.419003 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.418544    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-df94489-vlxg7" event={"ID":"ddb8f374-e401-461b-8618-921fa7187864","Type":"ContainerDied","Data":"27e44d20a19bc27ba88f4d6127e82ba59e3a72df0e34527e346f425c7e577156"}
Jul 10 02:41:54.419104 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.419088    2576 scope.go:117] "RemoveContainer" containerID="8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696"
Jul 10 02:41:54.428105 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.428091    2576 scope.go:117] "RemoveContainer" containerID="8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696"
Jul 10 02:41:54.428356 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:41:54.428337    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696\": container with ID starting with 8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696 not found: ID does not exist" containerID="8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696"
Jul 10 02:41:54.428411 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.428367    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696"} err="failed to get container status \"8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696\": rpc error: code = NotFound desc = could not find container \"8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696\": container with ID starting with 8840ab0c6c180a2394b671b8a3d33b7ba47edc4e9d1e791adb8ccb86f34f5696 not found: ID does not exist"
Jul 10 02:41:54.439759 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.439728    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-df94489-vlxg7"]
Jul 10 02:41:54.443939 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.443918    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-df94489-vlxg7"]
Jul 10 02:41:54.676498 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.676406    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-794d795678-n48kg" podUID="18bc67ac-8d22-449e-85db-2668bd6ba455" containerName="console" containerID="cri-o://6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256" gracePeriod=15
Jul 10 02:41:54.801506 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.801240    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddb8f374-e401-461b-8618-921fa7187864" path="/var/lib/kubelet/pods/ddb8f374-e401-461b-8618-921fa7187864/volumes"
Jul 10 02:41:54.918963 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.918936    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:54.982323 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982284    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-trusted-ca-bundle\") pod \"18bc67ac-8d22-449e-85db-2668bd6ba455\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") "
Jul 10 02:41:54.982514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982345    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-oauth-serving-cert\") pod \"18bc67ac-8d22-449e-85db-2668bd6ba455\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") "
Jul 10 02:41:54.982514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982387    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-serving-cert\") pod \"18bc67ac-8d22-449e-85db-2668bd6ba455\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") "
Jul 10 02:41:54.982514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982416    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-oauth-config\") pod \"18bc67ac-8d22-449e-85db-2668bd6ba455\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") "
Jul 10 02:41:54.982514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982454    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-service-ca\") pod \"18bc67ac-8d22-449e-85db-2668bd6ba455\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") "
Jul 10 02:41:54.982514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982491    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-console-config\") pod \"18bc67ac-8d22-449e-85db-2668bd6ba455\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") "
Jul 10 02:41:54.982802 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982548    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8k68d\" (UniqueName: \"kubernetes.io/projected/18bc67ac-8d22-449e-85db-2668bd6ba455-kube-api-access-8k68d\") pod \"18bc67ac-8d22-449e-85db-2668bd6ba455\" (UID: \"18bc67ac-8d22-449e-85db-2668bd6ba455\") "
Jul 10 02:41:54.982886 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982856    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "18bc67ac-8d22-449e-85db-2668bd6ba455" (UID: "18bc67ac-8d22-449e-85db-2668bd6ba455"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:41:54.982936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982889    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-console-config" (OuterVolumeSpecName: "console-config") pod "18bc67ac-8d22-449e-85db-2668bd6ba455" (UID: "18bc67ac-8d22-449e-85db-2668bd6ba455"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:41:54.982936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982909    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-service-ca" (OuterVolumeSpecName: "service-ca") pod "18bc67ac-8d22-449e-85db-2668bd6ba455" (UID: "18bc67ac-8d22-449e-85db-2668bd6ba455"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:41:54.982936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.982869    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "18bc67ac-8d22-449e-85db-2668bd6ba455" (UID: "18bc67ac-8d22-449e-85db-2668bd6ba455"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:41:54.984570 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.984548    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "18bc67ac-8d22-449e-85db-2668bd6ba455" (UID: "18bc67ac-8d22-449e-85db-2668bd6ba455"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:41:54.984734 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.984715    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "18bc67ac-8d22-449e-85db-2668bd6ba455" (UID: "18bc67ac-8d22-449e-85db-2668bd6ba455"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:41:54.984973 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:54.984957    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18bc67ac-8d22-449e-85db-2668bd6ba455-kube-api-access-8k68d" (OuterVolumeSpecName: "kube-api-access-8k68d") pod "18bc67ac-8d22-449e-85db-2668bd6ba455" (UID: "18bc67ac-8d22-449e-85db-2668bd6ba455"). InnerVolumeSpecName "kube-api-access-8k68d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:41:55.083981 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.083946    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k68d\" (UniqueName: \"kubernetes.io/projected/18bc67ac-8d22-449e-85db-2668bd6ba455-kube-api-access-8k68d\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:55.083981 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.083977    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-trusted-ca-bundle\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:55.083981 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.083986    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-oauth-serving-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:55.084237 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.084017    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-serving-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:55.084237 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.084026    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18bc67ac-8d22-449e-85db-2668bd6ba455-console-oauth-config\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:55.084237 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.084036    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-service-ca\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:55.084237 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.084045    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18bc67ac-8d22-449e-85db-2668bd6ba455-console-config\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:41:55.425107 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.425023    2576 generic.go:356] "Generic (PLEG): container finished" podID="18bc67ac-8d22-449e-85db-2668bd6ba455" containerID="6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256" exitCode=0
Jul 10 02:41:55.425107 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.425069    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-794d795678-n48kg"
Jul 10 02:41:55.425561 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.425072    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-794d795678-n48kg" event={"ID":"18bc67ac-8d22-449e-85db-2668bd6ba455","Type":"ContainerDied","Data":"6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256"}
Jul 10 02:41:55.425561 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.425202    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-794d795678-n48kg" event={"ID":"18bc67ac-8d22-449e-85db-2668bd6ba455","Type":"ContainerDied","Data":"fafa414a081fc75a6f1bb8ab277667174e26035f0f380a4028f1fd67f27672b6"}
Jul 10 02:41:55.425561 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.425232    2576 scope.go:117] "RemoveContainer" containerID="6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256"
Jul 10 02:41:55.433960 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.433940    2576 scope.go:117] "RemoveContainer" containerID="6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256"
Jul 10 02:41:55.434197 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:41:55.434176    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256\": container with ID starting with 6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256 not found: ID does not exist" containerID="6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256"
Jul 10 02:41:55.434261 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.434203    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256"} err="failed to get container status \"6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256\": rpc error: code = NotFound desc = could not find container \"6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256\": container with ID starting with 6fc79e5d5757858bbc0240641f04a608abd197668ac77d50c44bc76f9fe52256 not found: ID does not exist"
Jul 10 02:41:55.447351 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.447325    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-794d795678-n48kg"]
Jul 10 02:41:55.454328 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:55.454306    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-794d795678-n48kg"]
Jul 10 02:41:56.798398 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:41:56.798360    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18bc67ac-8d22-449e-85db-2668bd6ba455" path="/var/lib/kubelet/pods/18bc67ac-8d22-449e-85db-2668bd6ba455/volumes"
Jul 10 02:42:20.296815 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:20.296778    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:20.312650 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:20.312626    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:20.551863 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:20.551732    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:34.125856 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.125813    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:42:34.126370 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.126191    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="alertmanager" containerID="cri-o://e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730" gracePeriod=120
Jul 10 02:42:34.126370 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.126295    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy-web" containerID="cri-o://2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb" gracePeriod=120
Jul 10 02:42:34.126499 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.126373    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="config-reloader" containerID="cri-o://c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66" gracePeriod=120
Jul 10 02:42:34.126499 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.126442    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy-metric" containerID="cri-o://eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b" gracePeriod=120
Jul 10 02:42:34.126499 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.126449    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="prom-label-proxy" containerID="cri-o://bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068" gracePeriod=120
Jul 10 02:42:34.126632 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.126595    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy" containerID="cri-o://be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2" gracePeriod=120
Jul 10 02:42:34.598586 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.598550    2576 generic.go:356] "Generic (PLEG): container finished" podID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerID="bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068" exitCode=0
Jul 10 02:42:34.598586 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.598576    2576 generic.go:356] "Generic (PLEG): container finished" podID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerID="be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2" exitCode=0
Jul 10 02:42:34.598586 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.598583    2576 generic.go:356] "Generic (PLEG): container finished" podID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerID="c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66" exitCode=0
Jul 10 02:42:34.598586 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.598591    2576 generic.go:356] "Generic (PLEG): container finished" podID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerID="e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730" exitCode=0
Jul 10 02:42:34.598954 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.598628    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerDied","Data":"bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068"}
Jul 10 02:42:34.598954 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.598714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerDied","Data":"be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2"}
Jul 10 02:42:34.598954 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.598777    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerDied","Data":"c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66"}
Jul 10 02:42:34.598954 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:34.598822    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerDied","Data":"e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730"}
Jul 10 02:42:35.372911 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.372884    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.534548 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534520    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-web-config\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534734 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534558    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534734 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534593    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7pfv\" (UniqueName: \"kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-kube-api-access-b7pfv\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534734 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534640    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-main-tls\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534734 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534691    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-cluster-tls-config\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534735    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-metrics-client-ca\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534786    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-volume\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534811    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-tls-assets\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534843    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-main-db\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534879    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534926    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.534980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.534958    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-trusted-ca-bundle\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.535309 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.535004    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-out\") pod \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\" (UID: \"4d16a64b-256e-4780-8e59-9ae1c636f8c9\") "
Jul 10 02:42:35.535309 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.535208    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:35.535420 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.535351    2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-main-db\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.535930 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.535898    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:35.535930 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.535907    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:35.537969 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.537915    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-volume" (OuterVolumeSpecName: "config-volume") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:35.538997 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.538969    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:35.539102 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.539025    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-kube-api-access-b7pfv" (OuterVolumeSpecName: "kube-api-access-b7pfv") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "kube-api-access-b7pfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:42:35.539102 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.539041    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:35.539276 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.539240    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-out" (OuterVolumeSpecName: "config-out") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:35.539416 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.539393    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:35.539490 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.539420    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:35.539571 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.539544    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:42:35.542686 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.542660    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:35.549305 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.549279    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-web-config" (OuterVolumeSpecName: "web-config") pod "4d16a64b-256e-4780-8e59-9ae1c636f8c9" (UID: "4d16a64b-256e-4780-8e59-9ae1c636f8c9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:35.605528 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.605491    2576 generic.go:356] "Generic (PLEG): container finished" podID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerID="eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b" exitCode=0
Jul 10 02:42:35.605528 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.605520    2576 generic.go:356] "Generic (PLEG): container finished" podID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerID="2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb" exitCode=0
Jul 10 02:42:35.605778 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.605548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerDied","Data":"eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b"}
Jul 10 02:42:35.605778 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.605573    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.605778 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.605619    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerDied","Data":"2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb"}
Jul 10 02:42:35.605778 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.605662    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d16a64b-256e-4780-8e59-9ae1c636f8c9","Type":"ContainerDied","Data":"57cc4a64816ba2c75fb6a9cf2da83759f68405dd94f0872bdee39c502c4c83be"}
Jul 10 02:42:35.605778 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.605689    2576 scope.go:117] "RemoveContainer" containerID="bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068"
Jul 10 02:42:35.613836 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.613817    2576 scope.go:117] "RemoveContainer" containerID="eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b"
Jul 10 02:42:35.621017 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.620996    2576 scope.go:117] "RemoveContainer" containerID="be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2"
Jul 10 02:42:35.628173 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.628159    2576 scope.go:117] "RemoveContainer" containerID="2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb"
Jul 10 02:42:35.630838 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.630818    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:42:35.636556 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636535    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:42:35.636861 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636841    2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-web-config\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.636909 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636870    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.636909 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636890    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7pfv\" (UniqueName: \"kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-kube-api-access-b7pfv\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.636978 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636909    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-main-tls\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.636978 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636925    2576 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-cluster-tls-config\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.636978 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636939    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-metrics-client-ca\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.636978 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636954    2576 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-volume\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.636978 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636967    2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d16a64b-256e-4780-8e59-9ae1c636f8c9-tls-assets\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.637123 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636979    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.637123 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.636994    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d16a64b-256e-4780-8e59-9ae1c636f8c9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.637123 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.637012    2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d16a64b-256e-4780-8e59-9ae1c636f8c9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.637123 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.637026    2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d16a64b-256e-4780-8e59-9ae1c636f8c9-config-out\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:35.637459 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.637444    2576 scope.go:117] "RemoveContainer" containerID="c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66"
Jul 10 02:42:35.644820 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.644798    2576 scope.go:117] "RemoveContainer" containerID="e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730"
Jul 10 02:42:35.652080 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.652057    2576 scope.go:117] "RemoveContainer" containerID="084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b"
Jul 10 02:42:35.660023 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.660004    2576 scope.go:117] "RemoveContainer" containerID="bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068"
Jul 10 02:42:35.660338 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:42:35.660318    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068\": container with ID starting with bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068 not found: ID does not exist" containerID="bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068"
Jul 10 02:42:35.660439 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.660350    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068"} err="failed to get container status \"bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068\": rpc error: code = NotFound desc = could not find container \"bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068\": container with ID starting with bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068 not found: ID does not exist"
Jul 10 02:42:35.660439 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.660370    2576 scope.go:117] "RemoveContainer" containerID="eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b"
Jul 10 02:42:35.660623 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:42:35.660606    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b\": container with ID starting with eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b not found: ID does not exist" containerID="eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b"
Jul 10 02:42:35.660664 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.660629    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b"} err="failed to get container status \"eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b\": rpc error: code = NotFound desc = could not find container \"eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b\": container with ID starting with eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b not found: ID does not exist"
Jul 10 02:42:35.660664 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.660644    2576 scope.go:117] "RemoveContainer" containerID="be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2"
Jul 10 02:42:35.660932 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:42:35.660910    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2\": container with ID starting with be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2 not found: ID does not exist" containerID="be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2"
Jul 10 02:42:35.660984 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.660931    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2"} err="failed to get container status \"be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2\": rpc error: code = NotFound desc = could not find container \"be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2\": container with ID starting with be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2 not found: ID does not exist"
Jul 10 02:42:35.660984 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.660944    2576 scope.go:117] "RemoveContainer" containerID="2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb"
Jul 10 02:42:35.661196 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:42:35.661181    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb\": container with ID starting with 2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb not found: ID does not exist" containerID="2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb"
Jul 10 02:42:35.661238 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.661200    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb"} err="failed to get container status \"2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb\": rpc error: code = NotFound desc = could not find container \"2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb\": container with ID starting with 2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb not found: ID does not exist"
Jul 10 02:42:35.661238 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.661212    2576 scope.go:117] "RemoveContainer" containerID="c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66"
Jul 10 02:42:35.661453 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:42:35.661436    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66\": container with ID starting with c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66 not found: ID does not exist" containerID="c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66"
Jul 10 02:42:35.661493 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.661459    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66"} err="failed to get container status \"c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66\": rpc error: code = NotFound desc = could not find container \"c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66\": container with ID starting with c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66 not found: ID does not exist"
Jul 10 02:42:35.661493 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.661475    2576 scope.go:117] "RemoveContainer" containerID="e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730"
Jul 10 02:42:35.661696 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:42:35.661681    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730\": container with ID starting with e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730 not found: ID does not exist" containerID="e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730"
Jul 10 02:42:35.661739 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.661699    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730"} err="failed to get container status \"e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730\": rpc error: code = NotFound desc = could not find container \"e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730\": container with ID starting with e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730 not found: ID does not exist"
Jul 10 02:42:35.661739 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.661714    2576 scope.go:117] "RemoveContainer" containerID="084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b"
Jul 10 02:42:35.661998 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:42:35.661976    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b\": container with ID starting with 084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b not found: ID does not exist" containerID="084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b"
Jul 10 02:42:35.662053 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.662002    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b"} err="failed to get container status \"084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b\": rpc error: code = NotFound desc = could not find container \"084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b\": container with ID starting with 084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b not found: ID does not exist"
Jul 10 02:42:35.662053 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.662016    2576 scope.go:117] "RemoveContainer" containerID="bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068"
Jul 10 02:42:35.662262 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.662246    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068"} err="failed to get container status \"bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068\": rpc error: code = NotFound desc = could not find container \"bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068\": container with ID starting with bcd43c234687e653a6e5c60b023eb13499e5bc146e78045f01094b9a2e7c9068 not found: ID does not exist"
Jul 10 02:42:35.662309 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.662272    2576 scope.go:117] "RemoveContainer" containerID="eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b"
Jul 10 02:42:35.662513 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.662496    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b"} err="failed to get container status \"eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b\": rpc error: code = NotFound desc = could not find container \"eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b\": container with ID starting with eddd6b919fda135f43d4f2904e8643bf44f562c72e6e3a999f80edc130cc8a7b not found: ID does not exist"
Jul 10 02:42:35.662561 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.662515    2576 scope.go:117] "RemoveContainer" containerID="be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2"
Jul 10 02:42:35.662774 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.662737    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2"} err="failed to get container status \"be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2\": rpc error: code = NotFound desc = could not find container \"be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2\": container with ID starting with be361bf386154d384eed3b1de0542ada4b1cc0ab9ebe1cc4b4e2284c3bb380a2 not found: ID does not exist"
Jul 10 02:42:35.662825 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.662775    2576 scope.go:117] "RemoveContainer" containerID="2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb"
Jul 10 02:42:35.663014 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.662997    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb"} err="failed to get container status \"2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb\": rpc error: code = NotFound desc = could not find container \"2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb\": container with ID starting with 2160c8763332890f42c65e14015d730a767f68172a6a2dd4893120c5668715eb not found: ID does not exist"
Jul 10 02:42:35.663014 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.663013    2576 scope.go:117] "RemoveContainer" containerID="c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66"
Jul 10 02:42:35.663203 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.663187    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66"} err="failed to get container status \"c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66\": rpc error: code = NotFound desc = could not find container \"c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66\": container with ID starting with c4a891484e6163234ba529a1669f993a7063b4e3ba8f823a86042e2b5a7cfc66 not found: ID does not exist"
Jul 10 02:42:35.663241 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.663202    2576 scope.go:117] "RemoveContainer" containerID="e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730"
Jul 10 02:42:35.663407 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.663390    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730"} err="failed to get container status \"e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730\": rpc error: code = NotFound desc = could not find container \"e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730\": container with ID starting with e3d273da7cd2646795bbc9404a146c558f8f096f2115dd4889d0400443eb1730 not found: ID does not exist"
Jul 10 02:42:35.663452 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.663408    2576 scope.go:117] "RemoveContainer" containerID="084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b"
Jul 10 02:42:35.663638 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.663622    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b"} err="failed to get container status \"084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b\": rpc error: code = NotFound desc = could not find container \"084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b\": container with ID starting with 084a0c0f86cb4266f28239e6507da2c66fabd9d98cb2242e1b96f2aa86be545b not found: ID does not exist"
Jul 10 02:42:35.667188 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667167    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:42:35.667467 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667454    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy-web"
Jul 10 02:42:35.667513 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667468    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy-web"
Jul 10 02:42:35.667513 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667477    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18bc67ac-8d22-449e-85db-2668bd6ba455" containerName="console"
Jul 10 02:42:35.667513 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667483    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="18bc67ac-8d22-449e-85db-2668bd6ba455" containerName="console"
Jul 10 02:42:35.667513 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667496    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy-metric"
Jul 10 02:42:35.667513 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667502    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy-metric"
Jul 10 02:42:35.667513 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667512    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667517    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667525    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="init-config-reloader"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667530    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="init-config-reloader"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667536    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="config-reloader"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667541    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="config-reloader"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667549    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="prom-label-proxy"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667554    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="prom-label-proxy"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667563    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddb8f374-e401-461b-8618-921fa7187864" containerName="registry"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667568    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddb8f374-e401-461b-8618-921fa7187864" containerName="registry"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667573    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="alertmanager"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667578    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="alertmanager"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667630    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="alertmanager"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667637    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="18bc67ac-8d22-449e-85db-2668bd6ba455" containerName="console"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667643    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667650    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddb8f374-e401-461b-8618-921fa7187864" containerName="registry"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667655    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="prom-label-proxy"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667661    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="config-reloader"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667666    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy-metric"
Jul 10 02:42:35.667677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.667671    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" containerName="kube-rbac-proxy-web"
Jul 10 02:42:35.672033 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.672013    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.674774 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.674723    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 02:42:35.674974 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.674956    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 02:42:35.675052 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.674991    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tx2fg\""
Jul 10 02:42:35.675052 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.674961    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 02:42:35.675242 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.675060    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 02:42:35.675337 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.675322    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 02:42:35.675399 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.675336    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 02:42:35.675566 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.675552    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 02:42:35.675822 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.675810    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 02:42:35.682451 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.682433    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 02:42:35.687897 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.687874    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:42:35.838492 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838412    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de855ae6-469b-412b-9658-aa3731d39890-config-out\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838492 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838448    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838492 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838699 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838497    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-config-volume\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838699 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838541    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838699 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de855ae6-469b-412b-9658-aa3731d39890-tls-assets\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838699 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgqf7\" (UniqueName: \"kubernetes.io/projected/de855ae6-469b-412b-9658-aa3731d39890-kube-api-access-xgqf7\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838699 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838638    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838699 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838657    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de855ae6-469b-412b-9658-aa3731d39890-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838699 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838686    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de855ae6-469b-412b-9658-aa3731d39890-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838934 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838934 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838806    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-web-config\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.838934 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.838828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de855ae6-469b-412b-9658-aa3731d39890-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.939365 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.939313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.939365 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.939373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de855ae6-469b-412b-9658-aa3731d39890-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.939567 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.939402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de855ae6-469b-412b-9658-aa3731d39890-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.939567 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.939452    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.939567 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.939498    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-web-config\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.939567 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.939522    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de855ae6-469b-412b-9658-aa3731d39890-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.939777 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.939678    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de855ae6-469b-412b-9658-aa3731d39890-config-out\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.939777 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.939723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.939907 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.939879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de855ae6-469b-412b-9658-aa3731d39890-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.939976 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.939953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.940040 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.940006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-config-volume\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.940095 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.940035    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.940147 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.940100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de855ae6-469b-412b-9658-aa3731d39890-tls-assets\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.940147 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.940129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgqf7\" (UniqueName: \"kubernetes.io/projected/de855ae6-469b-412b-9658-aa3731d39890-kube-api-access-xgqf7\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.940345 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.940323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de855ae6-469b-412b-9658-aa3731d39890-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.940474 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.940450    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de855ae6-469b-412b-9658-aa3731d39890-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.942737 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.942572    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de855ae6-469b-412b-9658-aa3731d39890-config-out\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.942737 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.942649    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-web-config\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.942737 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.942722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.942737 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.942730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.943001 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.942878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-config-volume\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.943104 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.943084    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.943263 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.943247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.943424 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.943402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de855ae6-469b-412b-9658-aa3731d39890-tls-assets\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.944649 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.944623    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de855ae6-469b-412b-9658-aa3731d39890-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.948581 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.948563    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgqf7\" (UniqueName: \"kubernetes.io/projected/de855ae6-469b-412b-9658-aa3731d39890-kube-api-access-xgqf7\") pod \"alertmanager-main-0\" (UID: \"de855ae6-469b-412b-9658-aa3731d39890\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:35.982715 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:35.982689    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:42:36.126710 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:36.126673    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:42:36.127512 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:42:36.127482    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde855ae6_469b_412b_9658_aa3731d39890.slice/crio-c93317299df3463c9321d81e48067598a90e527daf41a7a37197e01383faaac0 WatchSource:0}: Error finding container c93317299df3463c9321d81e48067598a90e527daf41a7a37197e01383faaac0: Status 404 returned error can't find the container with id c93317299df3463c9321d81e48067598a90e527daf41a7a37197e01383faaac0
Jul 10 02:42:36.611917 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:36.611882    2576 generic.go:356] "Generic (PLEG): container finished" podID="de855ae6-469b-412b-9658-aa3731d39890" containerID="1ab74ee9c10051f065296c3a9806418c77099180ee36050198296d65c88ac032" exitCode=0
Jul 10 02:42:36.612285 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:36.611965    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de855ae6-469b-412b-9658-aa3731d39890","Type":"ContainerDied","Data":"1ab74ee9c10051f065296c3a9806418c77099180ee36050198296d65c88ac032"}
Jul 10 02:42:36.612285 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:36.612028    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de855ae6-469b-412b-9658-aa3731d39890","Type":"ContainerStarted","Data":"c93317299df3463c9321d81e48067598a90e527daf41a7a37197e01383faaac0"}
Jul 10 02:42:36.798806 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:36.798777    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d16a64b-256e-4780-8e59-9ae1c636f8c9" path="/var/lib/kubelet/pods/4d16a64b-256e-4780-8e59-9ae1c636f8c9/volumes"
Jul 10 02:42:37.618901 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:37.618864    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de855ae6-469b-412b-9658-aa3731d39890","Type":"ContainerStarted","Data":"e299469cf05d218d440b17c83a5ccd694546d1fc4b14b6e7dc420e0ef4081b25"}
Jul 10 02:42:37.619277 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:37.618936    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de855ae6-469b-412b-9658-aa3731d39890","Type":"ContainerStarted","Data":"95148556851daca48e816ffde31cdaef4e8eafe2d2213e50ba72478128d52d3e"}
Jul 10 02:42:37.619277 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:37.618964    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de855ae6-469b-412b-9658-aa3731d39890","Type":"ContainerStarted","Data":"afbb3c5c8653eddd3e33b1d22102783f878e97ff8d048ce75dc915772a5395da"}
Jul 10 02:42:37.619277 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:37.618991    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de855ae6-469b-412b-9658-aa3731d39890","Type":"ContainerStarted","Data":"3106e62cef5cb50bae588d92b6517ac73d2334ecc1f0c2b5576ae3033b24683e"}
Jul 10 02:42:37.619277 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:37.619015    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de855ae6-469b-412b-9658-aa3731d39890","Type":"ContainerStarted","Data":"42b61d1a44342e741b7473e0fc8d7704b4a3adb5c809241a9835864b9bf0b1e7"}
Jul 10 02:42:37.619277 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:37.619038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de855ae6-469b-412b-9658-aa3731d39890","Type":"ContainerStarted","Data":"f08a79eb7143dd494d02b1a380a5a7c5033bbc29742531d9457c298b20b75600"}
Jul 10 02:42:37.648830 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:37.648773    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.631193801 podStartE2EDuration="2.64875856s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.564759ms" isStatefulPod=false podCreationTimestamp="2026-07-10 02:42:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:42:37.647138921 +0000 UTC m=+167.456404222" watchObservedRunningTime="2026-07-10 02:42:37.64875856 +0000 UTC m=+167.458023851"
Jul 10 02:42:38.477186 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.477143    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:42:38.477652 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.477537    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="prometheus" containerID="cri-o://bf68c2c7c8127af53e6177f3fd9970c9b3ae9c39e5044dc37af882c418f736a0" gracePeriod=600
Jul 10 02:42:38.477652 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.477564    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="thanos-sidecar" containerID="cri-o://53d5efb46305e33b33ee9ff07b63a986ee82b9f7c8bac20248def31c30c7e305" gracePeriod=600
Jul 10 02:42:38.477652 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.477579    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy-web" containerID="cri-o://a4d19492e0ced72800fa8811e07eff6ce4a61cf40bf1fcdcb78ff4ce4f880b0b" gracePeriod=600
Jul 10 02:42:38.477652 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.477582    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="config-reloader" containerID="cri-o://a0ea2d8993ebf236d4522c17f03f77cb02a43cd31a82b7ed293f9ede6a801700" gracePeriod=600
Jul 10 02:42:38.477652 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.477551    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy" containerID="cri-o://4246ba564a8fd60286aced061e9eee8a1f013683755903bced414043786b0108" gracePeriod=600
Jul 10 02:42:38.477988 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.477609    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://044dde645f38752db78d2825136a45309ff08383d3f016d665827175d4f6bf16" gracePeriod=600
Jul 10 02:42:38.627946 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.627913    2576 generic.go:356] "Generic (PLEG): container finished" podID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerID="044dde645f38752db78d2825136a45309ff08383d3f016d665827175d4f6bf16" exitCode=0
Jul 10 02:42:38.627946 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.627941    2576 generic.go:356] "Generic (PLEG): container finished" podID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerID="4246ba564a8fd60286aced061e9eee8a1f013683755903bced414043786b0108" exitCode=0
Jul 10 02:42:38.627946 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.627948    2576 generic.go:356] "Generic (PLEG): container finished" podID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerID="a4d19492e0ced72800fa8811e07eff6ce4a61cf40bf1fcdcb78ff4ce4f880b0b" exitCode=0
Jul 10 02:42:38.627946 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.627954    2576 generic.go:356] "Generic (PLEG): container finished" podID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerID="53d5efb46305e33b33ee9ff07b63a986ee82b9f7c8bac20248def31c30c7e305" exitCode=0
Jul 10 02:42:38.627946 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.627959    2576 generic.go:356] "Generic (PLEG): container finished" podID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerID="a0ea2d8993ebf236d4522c17f03f77cb02a43cd31a82b7ed293f9ede6a801700" exitCode=0
Jul 10 02:42:38.627946 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.627965    2576 generic.go:356] "Generic (PLEG): container finished" podID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerID="bf68c2c7c8127af53e6177f3fd9970c9b3ae9c39e5044dc37af882c418f736a0" exitCode=0
Jul 10 02:42:38.628438 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.627994    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerDied","Data":"044dde645f38752db78d2825136a45309ff08383d3f016d665827175d4f6bf16"}
Jul 10 02:42:38.628438 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.628100    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerDied","Data":"4246ba564a8fd60286aced061e9eee8a1f013683755903bced414043786b0108"}
Jul 10 02:42:38.628438 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.628147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerDied","Data":"a4d19492e0ced72800fa8811e07eff6ce4a61cf40bf1fcdcb78ff4ce4f880b0b"}
Jul 10 02:42:38.628438 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.628185    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerDied","Data":"53d5efb46305e33b33ee9ff07b63a986ee82b9f7c8bac20248def31c30c7e305"}
Jul 10 02:42:38.628438 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.628229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerDied","Data":"a0ea2d8993ebf236d4522c17f03f77cb02a43cd31a82b7ed293f9ede6a801700"}
Jul 10 02:42:38.628438 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.628264    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerDied","Data":"bf68c2c7c8127af53e6177f3fd9970c9b3ae9c39e5044dc37af882c418f736a0"}
Jul 10 02:42:38.741944 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.741906    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:38.869875 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.869829    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-476vv\" (UniqueName: \"kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-kube-api-access-476vv\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870071 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.869884    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-serving-certs-ca-bundle\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870071 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.869929    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-trusted-ca-bundle\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870071 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.869950    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-kubelet-serving-ca-bundle\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870071 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.869968    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-metrics-client-certs\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870071 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.869984    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-web-config\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870071 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870003    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-metrics-client-ca\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870275 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870147    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-0\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870275 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870189    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-db\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870275 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870250    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-tls-assets\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870364 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870277    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-tls\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870364 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870310    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870364 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870354    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-grpc-tls\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870450 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870379    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-1\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870482 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870444    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870482 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870470    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config-out\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870542 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870518    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870571 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870544    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-2\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870600 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870573    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-kube-rbac-proxy\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870603    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-thanos-prometheus-http-client-file\") pod \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\" (UID: \"2de78ec6-0c81-444a-b03c-c5c508e6e48b\") "
Jul 10 02:42:38.870967 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.870935    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:38.871178 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.871134    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:38.871472 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.871447    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:38.872075 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.872032    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:38.872996 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.872971    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:38.873145 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.873120    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-kube-api-access-476vv" (OuterVolumeSpecName: "kube-api-access-476vv") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "kube-api-access-476vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:42:38.873337 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.873313    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:38.873414 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.873388    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:38.873486 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.873459    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:38.873622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.873599    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config-out" (OuterVolumeSpecName: "config-out") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:38.874334 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.873977    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:38.874871 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.874849    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:38.875450 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.875412    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:38.875450 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.875428    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:38.875792 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.875769    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config" (OuterVolumeSpecName: "config") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:38.875909 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.875883    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:42:38.875909 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.875884    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:38.876059 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.875940    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:38.876059 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.876029    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:38.884067 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.884043    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-web-config" (OuterVolumeSpecName: "web-config") pod "2de78ec6-0c81-444a-b03c-c5c508e6e48b" (UID: "2de78ec6-0c81-444a-b03c-c5c508e6e48b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:42:38.971294 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971258    2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971294 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971285    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971294 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971297    2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-kube-rbac-proxy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971307    2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971317    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-476vv\" (UniqueName: \"kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-kube-api-access-476vv\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971326    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971335    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971344    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971354    2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-metrics-client-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971362    2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-web-config\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971370    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-configmap-metrics-client-ca\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971378    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971386    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-db\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971393    2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2de78ec6-0c81-444a-b03c-c5c508e6e48b-tls-assets\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971402    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971410    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971420    2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-grpc-tls\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971429    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2de78ec6-0c81-444a-b03c-c5c508e6e48b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971437    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2de78ec6-0c81-444a-b03c-c5c508e6e48b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:38.971515 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:38.971445    2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2de78ec6-0c81-444a-b03c-c5c508e6e48b-config-out\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:42:39.636115 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.636074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2de78ec6-0c81-444a-b03c-c5c508e6e48b","Type":"ContainerDied","Data":"be127db0493cdb6ca394384fa8e8e680550436960b3781b072d01486b1df067f"}
Jul 10 02:42:39.636115 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.636121    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.636571 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.636176    2576 scope.go:117] "RemoveContainer" containerID="044dde645f38752db78d2825136a45309ff08383d3f016d665827175d4f6bf16"
Jul 10 02:42:39.644415 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.644395    2576 scope.go:117] "RemoveContainer" containerID="4246ba564a8fd60286aced061e9eee8a1f013683755903bced414043786b0108"
Jul 10 02:42:39.651366 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.651349    2576 scope.go:117] "RemoveContainer" containerID="a4d19492e0ced72800fa8811e07eff6ce4a61cf40bf1fcdcb78ff4ce4f880b0b"
Jul 10 02:42:39.657798 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.657777    2576 scope.go:117] "RemoveContainer" containerID="53d5efb46305e33b33ee9ff07b63a986ee82b9f7c8bac20248def31c30c7e305"
Jul 10 02:42:39.660638 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.660614    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:42:39.664844 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.664821    2576 scope.go:117] "RemoveContainer" containerID="a0ea2d8993ebf236d4522c17f03f77cb02a43cd31a82b7ed293f9ede6a801700"
Jul 10 02:42:39.667584 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.667562    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:42:39.673205 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.673191    2576 scope.go:117] "RemoveContainer" containerID="bf68c2c7c8127af53e6177f3fd9970c9b3ae9c39e5044dc37af882c418f736a0"
Jul 10 02:42:39.680564 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.680546    2576 scope.go:117] "RemoveContainer" containerID="a057d8daea96344403dcf8c458d8286c87f34337726f7fdd888f48f0d8dc8af4"
Jul 10 02:42:39.726076 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726038    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:42:39.726417 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726400    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="config-reloader"
Jul 10 02:42:39.726469 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726419    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="config-reloader"
Jul 10 02:42:39.726469 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726428    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy-thanos"
Jul 10 02:42:39.726469 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726435    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy-thanos"
Jul 10 02:42:39.726469 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726444    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="init-config-reloader"
Jul 10 02:42:39.726469 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726450    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="init-config-reloader"
Jul 10 02:42:39.726469 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726456    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="thanos-sidecar"
Jul 10 02:42:39.726469 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726462    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="thanos-sidecar"
Jul 10 02:42:39.726469 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726470    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy-web"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726476    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy-web"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726490    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726495    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726505    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="prometheus"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726510    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="prometheus"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726560    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="thanos-sidecar"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726566    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="prometheus"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726573    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy-web"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726579    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy-thanos"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726586    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="kube-rbac-proxy"
Jul 10 02:42:39.726685 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.726592    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" containerName="config-reloader"
Jul 10 02:42:39.733151 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.733128    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.736087 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.736065    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 02:42:39.736215 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.736069    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 02:42:39.736435 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.736410    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 02:42:39.736631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.736616    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 02:42:39.737561 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.737543    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 02:42:39.738017 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.738000    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 02:42:39.738093 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.738052    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 02:42:39.738164 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.738146    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 02:42:39.738250 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.738235    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1j680n36t9m04\""
Jul 10 02:42:39.738723 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.738709    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 02:42:39.738828 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.738762    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2kq2j\""
Jul 10 02:42:39.738881 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.738870    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 02:42:39.738928 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.738915    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 02:42:39.738977 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.738929    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 02:42:39.738977 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.738940    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 02:42:39.739427 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.739403    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 02:42:39.743481 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.743460    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 02:42:39.745889 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.745843    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:42:39.878814 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.878775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.878965 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.878821    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.878965 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.878868    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.878965 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.878902    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/adabbf19-0686-4bf0-9d96-732f673a5fa1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.878965 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.878923    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.878965 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.878952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-config\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879224 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879040    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879224 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879081    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879224 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879122    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-web-config\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879224 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879224 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879375 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879244    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879375 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879375 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879285    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879375 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879304    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879375 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879509 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879509 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj2l6\" (UniqueName: \"kubernetes.io/projected/adabbf19-0686-4bf0-9d96-732f673a5fa1-kube-api-access-fj2l6\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879509 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.879509 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.879455    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/adabbf19-0686-4bf0-9d96-732f673a5fa1-config-out\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.980837 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.980798    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.980837 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.980840    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981041 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.980923    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-web-config\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981041 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.980953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981041 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.980976    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981041 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.980992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981041 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981041 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981336 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981336 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981336 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981336 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981169    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj2l6\" (UniqueName: \"kubernetes.io/projected/adabbf19-0686-4bf0-9d96-732f673a5fa1-kube-api-access-fj2l6\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981336 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981198    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981336 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/adabbf19-0686-4bf0-9d96-732f673a5fa1-config-out\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981336 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981257    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981336 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981336 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981727 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981354    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/adabbf19-0686-4bf0-9d96-732f673a5fa1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981727 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981379    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981727 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-config\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981731    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981785    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.981964 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.981950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.982160 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.982139    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.982403 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.982377    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.982631 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.982606    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.984010 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.983983    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-web-config\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.984010 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.984003    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.984467 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.984155    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.984467 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.984432    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/adabbf19-0686-4bf0-9d96-732f673a5fa1-config-out\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.985205 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.984900    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-config\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.985205 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.984983    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.985439 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.985415    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.986385 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.986356    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/adabbf19-0686-4bf0-9d96-732f673a5fa1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.986385 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.986370    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/adabbf19-0686-4bf0-9d96-732f673a5fa1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.986535 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.986515    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.986718 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.986703    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.986773 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.986704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.987561 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.987533    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/adabbf19-0686-4bf0-9d96-732f673a5fa1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:39.992925 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:39.992906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj2l6\" (UniqueName: \"kubernetes.io/projected/adabbf19-0686-4bf0-9d96-732f673a5fa1-kube-api-access-fj2l6\") pod \"prometheus-k8s-0\" (UID: \"adabbf19-0686-4bf0-9d96-732f673a5fa1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:40.044130 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:40.044056    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:42:40.186505 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:40.186469    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:42:40.190260 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:42:40.190233    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadabbf19_0686_4bf0_9d96_732f673a5fa1.slice/crio-114fee49498e21f6e41730a62639f5f124db759bc5a9428eb816a74d2201335f WatchSource:0}: Error finding container 114fee49498e21f6e41730a62639f5f124db759bc5a9428eb816a74d2201335f: Status 404 returned error can't find the container with id 114fee49498e21f6e41730a62639f5f124db759bc5a9428eb816a74d2201335f
Jul 10 02:42:40.642606 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:40.642571    2576 generic.go:356] "Generic (PLEG): container finished" podID="adabbf19-0686-4bf0-9d96-732f673a5fa1" containerID="57170b503a2b7caf8c39dee8a44593300f69055b8ada5a563e78d8693d3dec83" exitCode=0
Jul 10 02:42:40.643004 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:40.642660    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"adabbf19-0686-4bf0-9d96-732f673a5fa1","Type":"ContainerDied","Data":"57170b503a2b7caf8c39dee8a44593300f69055b8ada5a563e78d8693d3dec83"}
Jul 10 02:42:40.643004 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:40.642778    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"adabbf19-0686-4bf0-9d96-732f673a5fa1","Type":"ContainerStarted","Data":"114fee49498e21f6e41730a62639f5f124db759bc5a9428eb816a74d2201335f"}
Jul 10 02:42:40.802882 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:40.802847    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2de78ec6-0c81-444a-b03c-c5c508e6e48b" path="/var/lib/kubelet/pods/2de78ec6-0c81-444a-b03c-c5c508e6e48b/volumes"
Jul 10 02:42:41.650278 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:41.650242    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"adabbf19-0686-4bf0-9d96-732f673a5fa1","Type":"ContainerStarted","Data":"87eef3743e1c31d6340ce0d701d096c458582a39ee6e6f9412a62c24238cf787"}
Jul 10 02:42:41.650672 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:41.650309    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"adabbf19-0686-4bf0-9d96-732f673a5fa1","Type":"ContainerStarted","Data":"93b002458cb7378aa1a06ec427faf28010cd513db45b7faebe5cb7799fa1e027"}
Jul 10 02:42:41.650672 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:41.650334    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"adabbf19-0686-4bf0-9d96-732f673a5fa1","Type":"ContainerStarted","Data":"0ddd576ef61c75d1ee3eb1f4319f3b7eeaccbe07d1ecc52a46bc1cf51dddee92"}
Jul 10 02:42:41.650672 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:41.650359    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"adabbf19-0686-4bf0-9d96-732f673a5fa1","Type":"ContainerStarted","Data":"646331ecf3ed38c3d72678de61f4f7e0d2b3cd83c0917d4b0d557593fc4100fe"}
Jul 10 02:42:41.650672 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:41.650382    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"adabbf19-0686-4bf0-9d96-732f673a5fa1","Type":"ContainerStarted","Data":"732f7c15c0148abbb80ae5a198f3edcbdcc078978c318ffdfb68fdf27db8fcd6"}
Jul 10 02:42:41.650672 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:41.650404    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"adabbf19-0686-4bf0-9d96-732f673a5fa1","Type":"ContainerStarted","Data":"22c2652942a6936dc79305621e8fb8c25e5bfff8109206fbf0c3eec2b324197b"}
Jul 10 02:42:41.685393 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:41.685333    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.667883186 podStartE2EDuration="2.685314237s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.431051ms" isStatefulPod=false podCreationTimestamp="2026-07-10 02:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:42:41.683460898 +0000 UTC m=+171.492726199" watchObservedRunningTime="2026-07-10 02:42:41.685314237 +0000 UTC m=+171.494579538"
Jul 10 02:42:45.044489 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:42:45.044393    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:43:40.045012 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:43:40.044978    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:43:40.060239 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:43:40.060216    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:43:40.903872 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:43:40.903844    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:44:50.575201 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:44:50.575170    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:44:50.578556 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:44:50.578529    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 02:44:50.583936 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:44:50.583914    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:45:22.137409 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.137377    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-htqt8"]
Jul 10 02:45:22.140723 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.140706    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-htqt8"
Jul 10 02:45:22.143456 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.143437    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 10 02:45:22.143581 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.143478    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7fg67\""
Jul 10 02:45:22.144545 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.144531    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 10 02:45:22.148694 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.148677    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-htqt8"]
Jul 10 02:45:22.205985 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.205951    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/041913b6-dd28-40c2-af34-7ef25b9c33e2-bound-sa-token\") pod \"cert-manager-55b4d954d5-htqt8\" (UID: \"041913b6-dd28-40c2-af34-7ef25b9c33e2\") " pod="cert-manager/cert-manager-55b4d954d5-htqt8"
Jul 10 02:45:22.206134 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.206014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjz4k\" (UniqueName: \"kubernetes.io/projected/041913b6-dd28-40c2-af34-7ef25b9c33e2-kube-api-access-gjz4k\") pod \"cert-manager-55b4d954d5-htqt8\" (UID: \"041913b6-dd28-40c2-af34-7ef25b9c33e2\") " pod="cert-manager/cert-manager-55b4d954d5-htqt8"
Jul 10 02:45:22.307025 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.306994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjz4k\" (UniqueName: \"kubernetes.io/projected/041913b6-dd28-40c2-af34-7ef25b9c33e2-kube-api-access-gjz4k\") pod \"cert-manager-55b4d954d5-htqt8\" (UID: \"041913b6-dd28-40c2-af34-7ef25b9c33e2\") " pod="cert-manager/cert-manager-55b4d954d5-htqt8"
Jul 10 02:45:22.307199 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.307047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/041913b6-dd28-40c2-af34-7ef25b9c33e2-bound-sa-token\") pod \"cert-manager-55b4d954d5-htqt8\" (UID: \"041913b6-dd28-40c2-af34-7ef25b9c33e2\") " pod="cert-manager/cert-manager-55b4d954d5-htqt8"
Jul 10 02:45:22.322275 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.322252    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/041913b6-dd28-40c2-af34-7ef25b9c33e2-bound-sa-token\") pod \"cert-manager-55b4d954d5-htqt8\" (UID: \"041913b6-dd28-40c2-af34-7ef25b9c33e2\") " pod="cert-manager/cert-manager-55b4d954d5-htqt8"
Jul 10 02:45:22.322275 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.322262    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjz4k\" (UniqueName: \"kubernetes.io/projected/041913b6-dd28-40c2-af34-7ef25b9c33e2-kube-api-access-gjz4k\") pod \"cert-manager-55b4d954d5-htqt8\" (UID: \"041913b6-dd28-40c2-af34-7ef25b9c33e2\") " pod="cert-manager/cert-manager-55b4d954d5-htqt8"
Jul 10 02:45:22.462670 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.462589    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-htqt8"
Jul 10 02:45:22.589862 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.589829    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-htqt8"]
Jul 10 02:45:22.593225 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:45:22.593200    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod041913b6_dd28_40c2_af34_7ef25b9c33e2.slice/crio-cabe6fb2240e8188bc72e873c218a3e92fc6f64ccd07035ad065b0779508e857 WatchSource:0}: Error finding container cabe6fb2240e8188bc72e873c218a3e92fc6f64ccd07035ad065b0779508e857: Status 404 returned error can't find the container with id cabe6fb2240e8188bc72e873c218a3e92fc6f64ccd07035ad065b0779508e857
Jul 10 02:45:22.595252 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:22.595231    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:45:23.306511 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:23.306475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-htqt8" event={"ID":"041913b6-dd28-40c2-af34-7ef25b9c33e2","Type":"ContainerStarted","Data":"cabe6fb2240e8188bc72e873c218a3e92fc6f64ccd07035ad065b0779508e857"}
Jul 10 02:45:26.324899 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:26.324852    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-htqt8" event={"ID":"041913b6-dd28-40c2-af34-7ef25b9c33e2","Type":"ContainerStarted","Data":"51c41de6b2549fd486c49ade35c0811efc2fe9853c4de5b68955b90da2cc236f"}
Jul 10 02:45:26.342604 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:26.342560    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-htqt8" podStartSLOduration=1.117066812 podStartE2EDuration="4.342545886s" totalImagesPullingTime="3.225479074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:26.341351197 +0000 UTC m=+336.150616510" watchObservedRunningTime="2026-07-10 02:45:26.342545886 +0000 UTC m=+336.151811189"
Jul 10 02:45:50.008253 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.008169    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"]
Jul 10 02:45:50.012288 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.012265    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.015721 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.015695    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 02:45:50.015868 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.015731    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 02:45:50.016429 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.016407    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 02:45:50.016808 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.016789    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 02:45:50.016910 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.016875    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q69t7\""
Jul 10 02:45:50.039589 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.039566    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"]
Jul 10 02:45:50.151924 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.151891    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9962e03-c473-4429-a9f7-36a28710d2d4-webhook-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.151924 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.151925    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9962e03-c473-4429-a9f7-36a28710d2d4-tmp\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.152138 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.151960    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9962e03-c473-4429-a9f7-36a28710d2d4-manifests\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.152138 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.152058    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94wc7\" (UniqueName: \"kubernetes.io/projected/e9962e03-c473-4429-a9f7-36a28710d2d4-kube-api-access-94wc7\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.152138 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.152095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9962e03-c473-4429-a9f7-36a28710d2d4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.253337 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.253301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94wc7\" (UniqueName: \"kubernetes.io/projected/e9962e03-c473-4429-a9f7-36a28710d2d4-kube-api-access-94wc7\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.253547 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.253355    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9962e03-c473-4429-a9f7-36a28710d2d4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.253547 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.253393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9962e03-c473-4429-a9f7-36a28710d2d4-webhook-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.253547 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.253414    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9962e03-c473-4429-a9f7-36a28710d2d4-tmp\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.253547 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.253463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9962e03-c473-4429-a9f7-36a28710d2d4-manifests\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.253950 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.253929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9962e03-c473-4429-a9f7-36a28710d2d4-manifests\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.255853 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.255827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9962e03-c473-4429-a9f7-36a28710d2d4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.256158 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.256137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9962e03-c473-4429-a9f7-36a28710d2d4-tmp\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.256287 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.256269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9962e03-c473-4429-a9f7-36a28710d2d4-webhook-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.279298 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.279233    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94wc7\" (UniqueName: \"kubernetes.io/projected/e9962e03-c473-4429-a9f7-36a28710d2d4-kube-api-access-94wc7\") pod \"opendatahub-operator-controller-manager-68647b69dd-r2qwd\" (UID: \"e9962e03-c473-4429-a9f7-36a28710d2d4\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.323163 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.323132    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:50.470975 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:50.470951    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"]
Jul 10 02:45:50.473281 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:45:50.473254    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9962e03_c473_4429_a9f7_36a28710d2d4.slice/crio-bf11c20b4c5fba07ab154191c344cf251fa3c6316b972eed3ec12b016f95b52a WatchSource:0}: Error finding container bf11c20b4c5fba07ab154191c344cf251fa3c6316b972eed3ec12b016f95b52a: Status 404 returned error can't find the container with id bf11c20b4c5fba07ab154191c344cf251fa3c6316b972eed3ec12b016f95b52a
Jul 10 02:45:51.431365 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:51.431310    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd" event={"ID":"e9962e03-c473-4429-a9f7-36a28710d2d4","Type":"ContainerStarted","Data":"bf11c20b4c5fba07ab154191c344cf251fa3c6316b972eed3ec12b016f95b52a"}
Jul 10 02:45:54.448817 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:54.448781    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd" event={"ID":"e9962e03-c473-4429-a9f7-36a28710d2d4","Type":"ContainerStarted","Data":"211549d8baba6ae31e5e977a26714f0d078ac47c8d845c64b401153708b314e5"}
Jul 10 02:45:55.454640 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:55.454603    2576 generic.go:356] "Generic (PLEG): container finished" podID="e9962e03-c473-4429-a9f7-36a28710d2d4" containerID="211549d8baba6ae31e5e977a26714f0d078ac47c8d845c64b401153708b314e5" exitCode=0
Jul 10 02:45:55.455123 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:55.454646    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd" event={"ID":"e9962e03-c473-4429-a9f7-36a28710d2d4","Type":"ContainerDied","Data":"211549d8baba6ae31e5e977a26714f0d078ac47c8d845c64b401153708b314e5"}
Jul 10 02:45:56.461384 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:56.461347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd" event={"ID":"e9962e03-c473-4429-a9f7-36a28710d2d4","Type":"ContainerStarted","Data":"6f3514558bb019bd9f453bf54f12c26e4f67836c66fe4d43f68c041897b8996c"}
Jul 10 02:45:56.461884 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:56.461480    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:45:56.487399 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:45:56.487338    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd" podStartSLOduration=2.9155346189999998 podStartE2EDuration="7.487322204s" totalImagesPullingTime="3.482265706s" totalInitContainerRuntime="1.089521879s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:56.484786582 +0000 UTC m=+366.294051887" watchObservedRunningTime="2026-07-10 02:45:56.487322204 +0000 UTC m=+366.296587505"
Jul 10 02:46:07.469015 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:07.468978    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-r2qwd"
Jul 10 02:46:18.653026 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.652992    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"]
Jul 10 02:46:18.659099 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.659077    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.662112 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.662089    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 02:46:18.662216 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.662107    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-h6mt8\""
Jul 10 02:46:18.671044 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.671021    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"]
Jul 10 02:46:18.806721 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.806684    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.806721 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.806722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.806980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.806770    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.806980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.806793    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djt7g\" (UniqueName: \"kubernetes.io/projected/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-kube-api-access-djt7g\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.806980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.806902    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.806980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.806941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.807174 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.807004    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.807174 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.807050    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.807174 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.807094    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.908533 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.908435    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.908707 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.908558    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.909622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.908825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.909622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.908867    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.909622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.908964    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.909622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.909000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djt7g\" (UniqueName: \"kubernetes.io/projected/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-kube-api-access-djt7g\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.909622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.909046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.909622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.909192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.909622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.909244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.909622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.909328    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.909622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.909371    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.909622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.909384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.910522 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.909689    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.910522 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.910053    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.911219 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.911197    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.911861 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.911844    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.921970 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.921944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djt7g\" (UniqueName: \"kubernetes.io/projected/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-kube-api-access-djt7g\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.922997 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.922973    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9b0605-5ed4-4f3f-ad3e-748978c988e1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dv6pn\" (UID: \"6d9b0605-5ed4-4f3f-ad3e-748978c988e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:18.970101 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:18.970065    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:19.184990 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:46:19.184948    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9b0605_5ed4_4f3f_ad3e_748978c988e1.slice/crio-3c3f0ab7a9f1cfd17a96ac585b2f93d2cf5a8154cc7c9af566ffee8dc5d5b766 WatchSource:0}: Error finding container 3c3f0ab7a9f1cfd17a96ac585b2f93d2cf5a8154cc7c9af566ffee8dc5d5b766: Status 404 returned error can't find the container with id 3c3f0ab7a9f1cfd17a96ac585b2f93d2cf5a8154cc7c9af566ffee8dc5d5b766
Jul 10 02:46:19.187941 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.187915    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"]
Jul 10 02:46:19.562605 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.562570    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn" event={"ID":"6d9b0605-5ed4-4f3f-ad3e-748978c988e1","Type":"ContainerStarted","Data":"3c3f0ab7a9f1cfd17a96ac585b2f93d2cf5a8154cc7c9af566ffee8dc5d5b766"}
Jul 10 02:46:19.799672 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.799639    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"]
Jul 10 02:46:19.804143 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.804121    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:19.807278 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.807258    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-x7tbt\""
Jul 10 02:46:19.808007 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.807981    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 02:46:19.808136 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.808114    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 10 02:46:19.808195 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.808127    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 10 02:46:19.808772 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.808735    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:46:19.809244 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.809231    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 10 02:46:19.826258 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.826206    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"]
Jul 10 02:46:19.916845 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.916811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmptn\" (UniqueName: \"kubernetes.io/projected/d15c3296-a7fc-4503-a551-a10ce7fcaa03-kube-api-access-bmptn\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:19.917025 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.916861    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d15c3296-a7fc-4503-a551-a10ce7fcaa03-manager-config\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:19.917095 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.917055    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d15c3296-a7fc-4503-a551-a10ce7fcaa03-metrics-cert\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:19.917154 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:19.917129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d15c3296-a7fc-4503-a551-a10ce7fcaa03-cert\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:20.018184 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.018153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmptn\" (UniqueName: \"kubernetes.io/projected/d15c3296-a7fc-4503-a551-a10ce7fcaa03-kube-api-access-bmptn\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:20.018349 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.018189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d15c3296-a7fc-4503-a551-a10ce7fcaa03-manager-config\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:20.018349 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.018246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d15c3296-a7fc-4503-a551-a10ce7fcaa03-metrics-cert\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:20.018349 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.018282    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d15c3296-a7fc-4503-a551-a10ce7fcaa03-cert\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:20.018967 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.018943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d15c3296-a7fc-4503-a551-a10ce7fcaa03-manager-config\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:20.020883 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.020863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d15c3296-a7fc-4503-a551-a10ce7fcaa03-metrics-cert\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:20.020994 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.020964    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d15c3296-a7fc-4503-a551-a10ce7fcaa03-cert\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:20.062114 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.062091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmptn\" (UniqueName: \"kubernetes.io/projected/d15c3296-a7fc-4503-a551-a10ce7fcaa03-kube-api-access-bmptn\") pod \"lws-controller-manager-b767bdcf8-frd25\" (UID: \"d15c3296-a7fc-4503-a551-a10ce7fcaa03\") " pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:20.114545 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.114478    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:20.285236 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.285213    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"]
Jul 10 02:46:20.288143 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:46:20.288117    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd15c3296_a7fc_4503_a551_a10ce7fcaa03.slice/crio-cb12703df0744c54d71c733e7d42366ec650ad2346b1575190df301de3c6cc06 WatchSource:0}: Error finding container cb12703df0744c54d71c733e7d42366ec650ad2346b1575190df301de3c6cc06: Status 404 returned error can't find the container with id cb12703df0744c54d71c733e7d42366ec650ad2346b1575190df301de3c6cc06
Jul 10 02:46:20.568598 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:20.568561    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25" event={"ID":"d15c3296-a7fc-4503-a551-a10ce7fcaa03","Type":"ContainerStarted","Data":"cb12703df0744c54d71c733e7d42366ec650ad2346b1575190df301de3c6cc06"}
Jul 10 02:46:22.135944 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.135910    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"]
Jul 10 02:46:22.141718 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.141682    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:22.144387 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.144359    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-nplx6\""
Jul 10 02:46:22.144514 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.144359    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 10 02:46:22.151721 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.151688    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"]
Jul 10 02:46:22.235015 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.234977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxwlj\" (UniqueName: \"kubernetes.io/projected/527b17dc-ebab-46c3-9987-919b9b63f0b9-kube-api-access-fxwlj\") pod \"kserve-controller-manager-d6f4b65b8-g4q8p\" (UID: \"527b17dc-ebab-46c3-9987-919b9b63f0b9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:22.235204 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.235174    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/527b17dc-ebab-46c3-9987-919b9b63f0b9-cert\") pod \"kserve-controller-manager-d6f4b65b8-g4q8p\" (UID: \"527b17dc-ebab-46c3-9987-919b9b63f0b9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:22.335869 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.335791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/527b17dc-ebab-46c3-9987-919b9b63f0b9-cert\") pod \"kserve-controller-manager-d6f4b65b8-g4q8p\" (UID: \"527b17dc-ebab-46c3-9987-919b9b63f0b9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:22.335869 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.335859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxwlj\" (UniqueName: \"kubernetes.io/projected/527b17dc-ebab-46c3-9987-919b9b63f0b9-kube-api-access-fxwlj\") pod \"kserve-controller-manager-d6f4b65b8-g4q8p\" (UID: \"527b17dc-ebab-46c3-9987-919b9b63f0b9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:22.336075 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:46:22.335960    2576 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 10 02:46:22.336075 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:46:22.336041    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527b17dc-ebab-46c3-9987-919b9b63f0b9-cert podName:527b17dc-ebab-46c3-9987-919b9b63f0b9 nodeName:}" failed. No retries permitted until 2026-07-10 02:46:22.836023318 +0000 UTC m=+392.645288602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/527b17dc-ebab-46c3-9987-919b9b63f0b9-cert") pod "kserve-controller-manager-d6f4b65b8-g4q8p" (UID: "527b17dc-ebab-46c3-9987-919b9b63f0b9") : secret "kserve-webhook-server-cert" not found
Jul 10 02:46:22.349034 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.349006    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxwlj\" (UniqueName: \"kubernetes.io/projected/527b17dc-ebab-46c3-9987-919b9b63f0b9-kube-api-access-fxwlj\") pod \"kserve-controller-manager-d6f4b65b8-g4q8p\" (UID: \"527b17dc-ebab-46c3-9987-919b9b63f0b9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:22.841330 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:22.841297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/527b17dc-ebab-46c3-9987-919b9b63f0b9-cert\") pod \"kserve-controller-manager-d6f4b65b8-g4q8p\" (UID: \"527b17dc-ebab-46c3-9987-919b9b63f0b9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:22.841487 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:46:22.841441    2576 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 10 02:46:22.841547 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:46:22.841509    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527b17dc-ebab-46c3-9987-919b9b63f0b9-cert podName:527b17dc-ebab-46c3-9987-919b9b63f0b9 nodeName:}" failed. No retries permitted until 2026-07-10 02:46:23.841493332 +0000 UTC m=+393.650758626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/527b17dc-ebab-46c3-9987-919b9b63f0b9-cert") pod "kserve-controller-manager-d6f4b65b8-g4q8p" (UID: "527b17dc-ebab-46c3-9987-919b9b63f0b9") : secret "kserve-webhook-server-cert" not found
Jul 10 02:46:23.853701 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:23.853394    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/527b17dc-ebab-46c3-9987-919b9b63f0b9-cert\") pod \"kserve-controller-manager-d6f4b65b8-g4q8p\" (UID: \"527b17dc-ebab-46c3-9987-919b9b63f0b9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:23.857413 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:23.857341    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/527b17dc-ebab-46c3-9987-919b9b63f0b9-cert\") pod \"kserve-controller-manager-d6f4b65b8-g4q8p\" (UID: \"527b17dc-ebab-46c3-9987-919b9b63f0b9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:23.954875 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:23.954816    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:24.047494 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.047449    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 02:46:24.047627 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.047521    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 02:46:24.047627 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.047549    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 02:46:24.213341 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.213303    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"]
Jul 10 02:46:24.216088 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:46:24.216055    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527b17dc_ebab_46c3_9987_919b9b63f0b9.slice/crio-106b80c0444f26f44a9cb0b4f67e78e6463f856a5da1ca58f0642e9f9dc89769 WatchSource:0}: Error finding container 106b80c0444f26f44a9cb0b4f67e78e6463f856a5da1ca58f0642e9f9dc89769: Status 404 returned error can't find the container with id 106b80c0444f26f44a9cb0b4f67e78e6463f856a5da1ca58f0642e9f9dc89769
Jul 10 02:46:24.590369 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.590332    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn" event={"ID":"6d9b0605-5ed4-4f3f-ad3e-748978c988e1","Type":"ContainerStarted","Data":"176b86035fed297c3832f98718e4bf7757ee5b96c04886181a3735da8703dccd"}
Jul 10 02:46:24.591925 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.591900    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25" event={"ID":"d15c3296-a7fc-4503-a551-a10ce7fcaa03","Type":"ContainerStarted","Data":"536d847b17c2a8706b5d75a01ef5700925a57fc987416b2cd018ef7c01faf6f3"}
Jul 10 02:46:24.592054 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.592000    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:24.593022 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.593001    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p" event={"ID":"527b17dc-ebab-46c3-9987-919b9b63f0b9","Type":"ContainerStarted","Data":"106b80c0444f26f44a9cb0b4f67e78e6463f856a5da1ca58f0642e9f9dc89769"}
Jul 10 02:46:24.614483 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.614445    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn" podStartSLOduration=1.754534424 podStartE2EDuration="6.614429815s" totalImagesPullingTime="4.859895391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:46:24.61272384 +0000 UTC m=+394.421989145" watchObservedRunningTime="2026-07-10 02:46:24.614429815 +0000 UTC m=+394.423695116"
Jul 10 02:46:24.632087 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.632043    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25" podStartSLOduration=1.869779892 podStartE2EDuration="5.632030637s" totalImagesPullingTime="3.762250745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:46:24.630551222 +0000 UTC m=+394.439816524" watchObservedRunningTime="2026-07-10 02:46:24.632030637 +0000 UTC m=+394.441295939"
Jul 10 02:46:24.970190 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:24.970152    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:25.976588 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:25.976559    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:26.604222 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:26.604170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p" event={"ID":"527b17dc-ebab-46c3-9987-919b9b63f0b9","Type":"ContainerStarted","Data":"4d58c4da2c45443d4bcd0492e7f6e4366704a24858b66bc5782f4c4b8e80ec47"}
Jul 10 02:46:26.604412 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:26.604245    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:46:26.604666 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:26.604644    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:26.605619 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:26.605598    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dv6pn"
Jul 10 02:46:26.623407 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:26.623357    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p" podStartSLOduration=2.3613079790000002 podStartE2EDuration="4.623341466s" totalImagesPullingTime="2.262033487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:46:26.621962642 +0000 UTC m=+396.431227945" watchObservedRunningTime="2026-07-10 02:46:26.623341466 +0000 UTC m=+396.432606768"
Jul 10 02:46:35.600016 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:35.599979    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-b767bdcf8-frd25"
Jul 10 02:46:57.613687 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:46:57.613658    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4q8p"
Jul 10 02:47:11.530952 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:11.530913    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8lq8"]
Jul 10 02:47:11.543213 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:11.543188    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c8lq8"
Jul 10 02:47:11.546117 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:11.546091    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 02:47:11.546486 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:11.546467    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 02:47:11.547064 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:11.547039    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8lq8"]
Jul 10 02:47:11.678359 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:11.678328    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vxft\" (UniqueName: \"kubernetes.io/projected/9b161336-a827-499e-a404-2e89fa7682ac-kube-api-access-7vxft\") pod \"kuadrant-operator-catalog-c8lq8\" (UID: \"9b161336-a827-499e-a404-2e89fa7682ac\") " pod="kuadrant-system/kuadrant-operator-catalog-c8lq8"
Jul 10 02:47:11.779528 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:11.779496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vxft\" (UniqueName: \"kubernetes.io/projected/9b161336-a827-499e-a404-2e89fa7682ac-kube-api-access-7vxft\") pod \"kuadrant-operator-catalog-c8lq8\" (UID: \"9b161336-a827-499e-a404-2e89fa7682ac\") " pod="kuadrant-system/kuadrant-operator-catalog-c8lq8"
Jul 10 02:47:11.788550 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:11.788484    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vxft\" (UniqueName: \"kubernetes.io/projected/9b161336-a827-499e-a404-2e89fa7682ac-kube-api-access-7vxft\") pod \"kuadrant-operator-catalog-c8lq8\" (UID: \"9b161336-a827-499e-a404-2e89fa7682ac\") " pod="kuadrant-system/kuadrant-operator-catalog-c8lq8"
Jul 10 02:47:11.855031 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:11.854980    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c8lq8"
Jul 10 02:47:11.990246 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:11.990219    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8lq8"]
Jul 10 02:47:11.993848 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:47:11.993807    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b161336_a827_499e_a404_2e89fa7682ac.slice/crio-681e283885166dfedd9cb32bbdc4b5f4ba431d39232c853d0dcc21dbdbc918a9 WatchSource:0}: Error finding container 681e283885166dfedd9cb32bbdc4b5f4ba431d39232c853d0dcc21dbdbc918a9: Status 404 returned error can't find the container with id 681e283885166dfedd9cb32bbdc4b5f4ba431d39232c853d0dcc21dbdbc918a9
Jul 10 02:47:12.110442 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.110362    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8lq8"]
Jul 10 02:47:12.130094 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.130034    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cw67h"]
Jul 10 02:47:12.134549 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.134533    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cw67h"
Jul 10 02:47:12.137110 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.137094    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9w849\""
Jul 10 02:47:12.141806 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.141621    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cw67h"]
Jul 10 02:47:12.284045 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.284002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmk44\" (UniqueName: \"kubernetes.io/projected/4f80dac4-aff7-4544-8056-1479d748a33d-kube-api-access-xmk44\") pod \"kuadrant-operator-catalog-cw67h\" (UID: \"4f80dac4-aff7-4544-8056-1479d748a33d\") " pod="kuadrant-system/kuadrant-operator-catalog-cw67h"
Jul 10 02:47:12.384569 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.384486    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmk44\" (UniqueName: \"kubernetes.io/projected/4f80dac4-aff7-4544-8056-1479d748a33d-kube-api-access-xmk44\") pod \"kuadrant-operator-catalog-cw67h\" (UID: \"4f80dac4-aff7-4544-8056-1479d748a33d\") " pod="kuadrant-system/kuadrant-operator-catalog-cw67h"
Jul 10 02:47:12.393979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.393951    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmk44\" (UniqueName: \"kubernetes.io/projected/4f80dac4-aff7-4544-8056-1479d748a33d-kube-api-access-xmk44\") pod \"kuadrant-operator-catalog-cw67h\" (UID: \"4f80dac4-aff7-4544-8056-1479d748a33d\") " pod="kuadrant-system/kuadrant-operator-catalog-cw67h"
Jul 10 02:47:12.444239 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.444207    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cw67h"
Jul 10 02:47:12.578102 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.578078    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cw67h"]
Jul 10 02:47:12.580548 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:47:12.580528    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f80dac4_aff7_4544_8056_1479d748a33d.slice/crio-6cd1419ca240a748de31aa4fd05d6b0d0551a6e7e8f801896a59f3ebf08dfcc1 WatchSource:0}: Error finding container 6cd1419ca240a748de31aa4fd05d6b0d0551a6e7e8f801896a59f3ebf08dfcc1: Status 404 returned error can't find the container with id 6cd1419ca240a748de31aa4fd05d6b0d0551a6e7e8f801896a59f3ebf08dfcc1
Jul 10 02:47:12.811590 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.811553    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c8lq8" event={"ID":"9b161336-a827-499e-a404-2e89fa7682ac","Type":"ContainerStarted","Data":"681e283885166dfedd9cb32bbdc4b5f4ba431d39232c853d0dcc21dbdbc918a9"}
Jul 10 02:47:12.812767 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:12.812721    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cw67h" event={"ID":"4f80dac4-aff7-4544-8056-1479d748a33d","Type":"ContainerStarted","Data":"6cd1419ca240a748de31aa4fd05d6b0d0551a6e7e8f801896a59f3ebf08dfcc1"}
Jul 10 02:47:14.825214 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:14.825171    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cw67h" event={"ID":"4f80dac4-aff7-4544-8056-1479d748a33d","Type":"ContainerStarted","Data":"4c1486bd59aa4c0e8972ca2b5012663ee49d875c8ba3575c6a0f0cad41a4f3fc"}
Jul 10 02:47:14.826691 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:14.826661    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c8lq8" event={"ID":"9b161336-a827-499e-a404-2e89fa7682ac","Type":"ContainerStarted","Data":"947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192"}
Jul 10 02:47:14.826847 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:14.826756    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-c8lq8" podUID="9b161336-a827-499e-a404-2e89fa7682ac" containerName="registry-server" containerID="cri-o://947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192" gracePeriod=2
Jul 10 02:47:14.842620 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:14.842577    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-cw67h" podStartSLOduration=1.296191496 podStartE2EDuration="2.842557847s" totalImagesPullingTime="1.546366351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:14.841934433 +0000 UTC m=+444.651199747" watchObservedRunningTime="2026-07-10 02:47:14.842557847 +0000 UTC m=+444.651823127"
Jul 10 02:47:14.859729 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:14.859684    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-c8lq8" podStartSLOduration=1.729812487 podStartE2EDuration="3.859668524s" totalImagesPullingTime="2.129856037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:14.858731493 +0000 UTC m=+444.667996798" watchObservedRunningTime="2026-07-10 02:47:14.859668524 +0000 UTC m=+444.668933826"
Jul 10 02:47:15.079402 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.079344    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c8lq8"
Jul 10 02:47:15.212033 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.211998    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vxft\" (UniqueName: \"kubernetes.io/projected/9b161336-a827-499e-a404-2e89fa7682ac-kube-api-access-7vxft\") pod \"9b161336-a827-499e-a404-2e89fa7682ac\" (UID: \"9b161336-a827-499e-a404-2e89fa7682ac\") "
Jul 10 02:47:15.214206 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.214180    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b161336-a827-499e-a404-2e89fa7682ac-kube-api-access-7vxft" (OuterVolumeSpecName: "kube-api-access-7vxft") pod "9b161336-a827-499e-a404-2e89fa7682ac" (UID: "9b161336-a827-499e-a404-2e89fa7682ac"). InnerVolumeSpecName "kube-api-access-7vxft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:15.312739 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.312701    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vxft\" (UniqueName: \"kubernetes.io/projected/9b161336-a827-499e-a404-2e89fa7682ac-kube-api-access-7vxft\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:47:15.832766 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.832702    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b161336-a827-499e-a404-2e89fa7682ac" containerID="947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192" exitCode=0
Jul 10 02:47:15.832766 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.832771    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c8lq8"
Jul 10 02:47:15.833364 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.832787    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c8lq8" event={"ID":"9b161336-a827-499e-a404-2e89fa7682ac","Type":"ContainerDied","Data":"947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192"}
Jul 10 02:47:15.833364 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.832845    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c8lq8" event={"ID":"9b161336-a827-499e-a404-2e89fa7682ac","Type":"ContainerDied","Data":"681e283885166dfedd9cb32bbdc4b5f4ba431d39232c853d0dcc21dbdbc918a9"}
Jul 10 02:47:15.833364 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.832876    2576 scope.go:117] "RemoveContainer" containerID="947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192"
Jul 10 02:47:15.842072 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.842046    2576 scope.go:117] "RemoveContainer" containerID="947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192"
Jul 10 02:47:15.842347 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:47:15.842327    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192\": container with ID starting with 947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192 not found: ID does not exist" containerID="947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192"
Jul 10 02:47:15.842415 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.842356    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192"} err="failed to get container status \"947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192\": rpc error: code = NotFound desc = could not find container \"947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192\": container with ID starting with 947065224f5deddf4da4b20cfef726ad4634e99644c034aa4cbc9b145dfa8192 not found: ID does not exist"
Jul 10 02:47:15.855880 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.855850    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8lq8"]
Jul 10 02:47:15.859885 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:15.859857    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8lq8"]
Jul 10 02:47:16.799132 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:16.799095    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b161336-a827-499e-a404-2e89fa7682ac" path="/var/lib/kubelet/pods/9b161336-a827-499e-a404-2e89fa7682ac/volumes"
Jul 10 02:47:22.445297 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:22.445263    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-cw67h"
Jul 10 02:47:22.445780 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:22.445316    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-cw67h"
Jul 10 02:47:22.467898 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:22.467871    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-cw67h"
Jul 10 02:47:22.886181 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:22.886155    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-cw67h"
Jul 10 02:47:43.953305 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.953266    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fdd69469c-lsvf9"]
Jul 10 02:47:43.953740 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.953663    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b161336-a827-499e-a404-2e89fa7682ac" containerName="registry-server"
Jul 10 02:47:43.953740 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.953677    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b161336-a827-499e-a404-2e89fa7682ac" containerName="registry-server"
Jul 10 02:47:43.953740 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.953728    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b161336-a827-499e-a404-2e89fa7682ac" containerName="registry-server"
Jul 10 02:47:43.957361 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.957333    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:43.960647 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.960626    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 10 02:47:43.963362 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.963126    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 10 02:47:43.963362 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.963155    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 10 02:47:43.963362 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.963156    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 10 02:47:43.963362 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.963213    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 10 02:47:43.963362 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.963160    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 10 02:47:43.963683 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.963381    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 10 02:47:43.964174 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.964152    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sfgqp\""
Jul 10 02:47:43.969683 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.969660    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 10 02:47:43.975371 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:43.975346    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fdd69469c-lsvf9"]
Jul 10 02:47:44.066608 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.066573    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-console-config\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.066807 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.066623    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-service-ca\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.066807 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.066651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-trusted-ca-bundle\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.066807 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.066679    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da7e1773-e30d-4956-82e7-6128cbcdf29f-console-oauth-config\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.066807 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.066696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da7e1773-e30d-4956-82e7-6128cbcdf29f-console-serving-cert\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.066945 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.066819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-oauth-serving-cert\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.066945 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.066852    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rffdv\" (UniqueName: \"kubernetes.io/projected/da7e1773-e30d-4956-82e7-6128cbcdf29f-kube-api-access-rffdv\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.110862 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.110819    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-p9l66"]
Jul 10 02:47:44.115258 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.115226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-p9l66"
Jul 10 02:47:44.118892 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.118875    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-2kmbr\""
Jul 10 02:47:44.130186 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.130162    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-p9l66"]
Jul 10 02:47:44.167658 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.167623    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-console-config\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.167897 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.167678    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-service-ca\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.167897 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.167705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-trusted-ca-bundle\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.167897 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.167798    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da7e1773-e30d-4956-82e7-6128cbcdf29f-console-oauth-config\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.167897 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.167826    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da7e1773-e30d-4956-82e7-6128cbcdf29f-console-serving-cert\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.167897 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.167865    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-oauth-serving-cert\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.168235 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.168046    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rffdv\" (UniqueName: \"kubernetes.io/projected/da7e1773-e30d-4956-82e7-6128cbcdf29f-kube-api-access-rffdv\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.168235 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.168131    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvhsb\" (UniqueName: \"kubernetes.io/projected/6c165bf8-331d-4fa9-a869-f5a4caaa5949-kube-api-access-fvhsb\") pod \"authorino-operator-75cc8d5cb7-p9l66\" (UID: \"6c165bf8-331d-4fa9-a869-f5a4caaa5949\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-p9l66"
Jul 10 02:47:44.168544 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.168519    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-console-config\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.168619 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.168519    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-service-ca\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.168619 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.168523    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-oauth-serving-cert\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.168700 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.168681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da7e1773-e30d-4956-82e7-6128cbcdf29f-trusted-ca-bundle\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.170689 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.170670    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da7e1773-e30d-4956-82e7-6128cbcdf29f-console-serving-cert\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.170742 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.170721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da7e1773-e30d-4956-82e7-6128cbcdf29f-console-oauth-config\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.180328 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.180305    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rffdv\" (UniqueName: \"kubernetes.io/projected/da7e1773-e30d-4956-82e7-6128cbcdf29f-kube-api-access-rffdv\") pod \"console-fdd69469c-lsvf9\" (UID: \"da7e1773-e30d-4956-82e7-6128cbcdf29f\") " pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.268006 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.267970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:44.269245 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.269219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvhsb\" (UniqueName: \"kubernetes.io/projected/6c165bf8-331d-4fa9-a869-f5a4caaa5949-kube-api-access-fvhsb\") pod \"authorino-operator-75cc8d5cb7-p9l66\" (UID: \"6c165bf8-331d-4fa9-a869-f5a4caaa5949\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-p9l66"
Jul 10 02:47:44.284831 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.284800    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvhsb\" (UniqueName: \"kubernetes.io/projected/6c165bf8-331d-4fa9-a869-f5a4caaa5949-kube-api-access-fvhsb\") pod \"authorino-operator-75cc8d5cb7-p9l66\" (UID: \"6c165bf8-331d-4fa9-a869-f5a4caaa5949\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-p9l66"
Jul 10 02:47:44.406725 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.406609    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fdd69469c-lsvf9"]
Jul 10 02:47:44.409051 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:47:44.409027    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda7e1773_e30d_4956_82e7_6128cbcdf29f.slice/crio-eb2ecae150be7eaa1e469744799debf61084c71528717c26760513f18c4cd5cb WatchSource:0}: Error finding container eb2ecae150be7eaa1e469744799debf61084c71528717c26760513f18c4cd5cb: Status 404 returned error can't find the container with id eb2ecae150be7eaa1e469744799debf61084c71528717c26760513f18c4cd5cb
Jul 10 02:47:44.425474 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.425450    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-p9l66"
Jul 10 02:47:44.570984 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.570906    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-p9l66"]
Jul 10 02:47:44.574595 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:47:44.574568    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c165bf8_331d_4fa9_a869_f5a4caaa5949.slice/crio-2a3c782154012570effd2b3fabdc7e029bb0ed5ac2a08243b19ad7f2b66c8fb7 WatchSource:0}: Error finding container 2a3c782154012570effd2b3fabdc7e029bb0ed5ac2a08243b19ad7f2b66c8fb7: Status 404 returned error can't find the container with id 2a3c782154012570effd2b3fabdc7e029bb0ed5ac2a08243b19ad7f2b66c8fb7
Jul 10 02:47:44.963142 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.963108    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-p9l66" event={"ID":"6c165bf8-331d-4fa9-a869-f5a4caaa5949","Type":"ContainerStarted","Data":"2a3c782154012570effd2b3fabdc7e029bb0ed5ac2a08243b19ad7f2b66c8fb7"}
Jul 10 02:47:44.964697 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.964664    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdd69469c-lsvf9" event={"ID":"da7e1773-e30d-4956-82e7-6128cbcdf29f","Type":"ContainerStarted","Data":"6f8594fd57e1cf46c82f83ef9967aed6b302b0f4fa95077146fcab1b44125f2e"}
Jul 10 02:47:44.964824 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.964713    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdd69469c-lsvf9" event={"ID":"da7e1773-e30d-4956-82e7-6128cbcdf29f","Type":"ContainerStarted","Data":"eb2ecae150be7eaa1e469744799debf61084c71528717c26760513f18c4cd5cb"}
Jul 10 02:47:44.989318 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:44.989274    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fdd69469c-lsvf9" podStartSLOduration=1.989259487 podStartE2EDuration="1.989259487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:44.985359691 +0000 UTC m=+474.794625005" watchObservedRunningTime="2026-07-10 02:47:44.989259487 +0000 UTC m=+474.798524834"
Jul 10 02:47:46.977304 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:46.977267    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-p9l66" event={"ID":"6c165bf8-331d-4fa9-a869-f5a4caaa5949","Type":"ContainerStarted","Data":"b95838154740f92702992776133580d8b2e01a5953cd00cbe15a8c1c734c8473"}
Jul 10 02:47:46.977687 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:46.977387    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-p9l66"
Jul 10 02:47:47.000029 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:46.999979    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-p9l66" podStartSLOduration=0.71792701 podStartE2EDuration="2.999951958s" totalImagesPullingTime="2.282024948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:46.998953009 +0000 UTC m=+476.808218312" watchObservedRunningTime="2026-07-10 02:47:46.999951958 +0000 UTC m=+476.809217318"
Jul 10 02:47:52.866622 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:52.866592    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"]
Jul 10 02:47:52.871859 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:52.871834    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:47:52.875496 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:52.875469    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-8ptbh\""
Jul 10 02:47:52.883766 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:52.882810    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"]
Jul 10 02:47:52.948980 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:52.948934    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7jxv\" (UniqueName: \"kubernetes.io/projected/9158063b-fe6d-41c8-b8d4-d866cf587a60-kube-api-access-n7jxv\") pod \"limitador-operator-controller-manager-859d7b78bd-znl76\" (UID: \"9158063b-fe6d-41c8-b8d4-d866cf587a60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:47:53.050437 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:53.050397    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7jxv\" (UniqueName: \"kubernetes.io/projected/9158063b-fe6d-41c8-b8d4-d866cf587a60-kube-api-access-n7jxv\") pod \"limitador-operator-controller-manager-859d7b78bd-znl76\" (UID: \"9158063b-fe6d-41c8-b8d4-d866cf587a60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:47:53.061071 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:53.061036    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7jxv\" (UniqueName: \"kubernetes.io/projected/9158063b-fe6d-41c8-b8d4-d866cf587a60-kube-api-access-n7jxv\") pod \"limitador-operator-controller-manager-859d7b78bd-znl76\" (UID: \"9158063b-fe6d-41c8-b8d4-d866cf587a60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:47:53.185144 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:53.185059    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:47:53.315789 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:53.315739    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"]
Jul 10 02:47:53.318143 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:47:53.318111    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9158063b_fe6d_41c8_b8d4_d866cf587a60.slice/crio-849a5e1c40a4b3d66bbcd808b7a3eebdcd5fc2400e692cc687c460ee6eb8b99f WatchSource:0}: Error finding container 849a5e1c40a4b3d66bbcd808b7a3eebdcd5fc2400e692cc687c460ee6eb8b99f: Status 404 returned error can't find the container with id 849a5e1c40a4b3d66bbcd808b7a3eebdcd5fc2400e692cc687c460ee6eb8b99f
Jul 10 02:47:54.012478 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:54.012444    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76" event={"ID":"9158063b-fe6d-41c8-b8d4-d866cf587a60","Type":"ContainerStarted","Data":"849a5e1c40a4b3d66bbcd808b7a3eebdcd5fc2400e692cc687c460ee6eb8b99f"}
Jul 10 02:47:54.268677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:54.268600    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:54.268857 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:54.268697    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:54.274345 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:54.274316    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:55.022709 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:55.022673    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fdd69469c-lsvf9"
Jul 10 02:47:56.024858 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:56.024823    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76" event={"ID":"9158063b-fe6d-41c8-b8d4-d866cf587a60","Type":"ContainerStarted","Data":"59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5"}
Jul 10 02:47:56.025255 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:56.025022    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:47:56.046065 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:56.046023    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76" podStartSLOduration=2.030723343 podStartE2EDuration="4.046008601s" totalImagesPullingTime="2.015285258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:56.044100837 +0000 UTC m=+485.853366143" watchObservedRunningTime="2026-07-10 02:47:56.046008601 +0000 UTC m=+485.855273904"
Jul 10 02:47:57.984701 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:47:57.984670    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-p9l66"
Jul 10 02:48:07.032291 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:07.032257    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:48:10.727344 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:10.727304    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"]
Jul 10 02:48:10.731042 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:10.731023    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:10.734402 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:10.734382    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4l7tn\""
Jul 10 02:48:10.747884 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:10.747856    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"]
Jul 10 02:48:10.913944 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:10.913910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2adba223-5fd4-4082-b2de-24db0ae1933a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8x2t\" (UID: \"2adba223-5fd4-4082-b2de-24db0ae1933a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:10.914109 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:10.914035    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqz6b\" (UniqueName: \"kubernetes.io/projected/2adba223-5fd4-4082-b2de-24db0ae1933a-kube-api-access-gqz6b\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8x2t\" (UID: \"2adba223-5fd4-4082-b2de-24db0ae1933a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:11.014868 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.014733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqz6b\" (UniqueName: \"kubernetes.io/projected/2adba223-5fd4-4082-b2de-24db0ae1933a-kube-api-access-gqz6b\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8x2t\" (UID: \"2adba223-5fd4-4082-b2de-24db0ae1933a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:11.014868 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.014874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2adba223-5fd4-4082-b2de-24db0ae1933a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8x2t\" (UID: \"2adba223-5fd4-4082-b2de-24db0ae1933a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:11.015239 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.015222    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2adba223-5fd4-4082-b2de-24db0ae1933a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8x2t\" (UID: \"2adba223-5fd4-4082-b2de-24db0ae1933a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:11.025656 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.025624    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqz6b\" (UniqueName: \"kubernetes.io/projected/2adba223-5fd4-4082-b2de-24db0ae1933a-kube-api-access-gqz6b\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8x2t\" (UID: \"2adba223-5fd4-4082-b2de-24db0ae1933a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:11.040880 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.040844    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:11.179840 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.179815    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"]
Jul 10 02:48:11.182970 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:48:11.182946    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2adba223_5fd4_4082_b2de_24db0ae1933a.slice/crio-b2f9bb3ff70374bf06810f9df186f9a88b2b8f96a947dac7f42df51fe615bfb7 WatchSource:0}: Error finding container b2f9bb3ff70374bf06810f9df186f9a88b2b8f96a947dac7f42df51fe615bfb7: Status 404 returned error can't find the container with id b2f9bb3ff70374bf06810f9df186f9a88b2b8f96a947dac7f42df51fe615bfb7
Jul 10 02:48:11.196977 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.196949    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"]
Jul 10 02:48:11.199483 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.199461    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"]
Jul 10 02:48:11.263937 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.263905    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"]
Jul 10 02:48:11.264119 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.264097    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" containerName="manager" containerID="cri-o://59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5" gracePeriod=2
Jul 10 02:48:11.267293 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:48:11.267217    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-znl76\" is forbidden: User \"system:node:ip-10-0-143-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-93.ec2.internal' and this object" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:48:11.269076 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.269053    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"]
Jul 10 02:48:11.290143 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.290118    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk"]
Jul 10 02:48:11.290671 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.290648    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" containerName="manager"
Jul 10 02:48:11.290671 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.290673    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" containerName="manager"
Jul 10 02:48:11.291180 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.290782    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" containerName="manager"
Jul 10 02:48:11.294440 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.294204    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk"
Jul 10 02:48:11.297580 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:48:11.297552    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-znl76\" is forbidden: User \"system:node:ip-10-0-143-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-93.ec2.internal' and this object" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:48:11.309483 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.309399    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk"]
Jul 10 02:48:11.420079 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.420034    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slvtd\" (UniqueName: \"kubernetes.io/projected/e7dd2af0-db13-4782-88d3-4afcadd2202b-kube-api-access-slvtd\") pod \"limitador-operator-controller-manager-859d7b78bd-h2jwk\" (UID: \"e7dd2af0-db13-4782-88d3-4afcadd2202b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk"
Jul 10 02:48:11.497589 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.497561    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:48:11.501190 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:48:11.501164    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-znl76\" is forbidden: User \"system:node:ip-10-0-143-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-93.ec2.internal' and this object" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:48:11.520692 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.520606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slvtd\" (UniqueName: \"kubernetes.io/projected/e7dd2af0-db13-4782-88d3-4afcadd2202b-kube-api-access-slvtd\") pod \"limitador-operator-controller-manager-859d7b78bd-h2jwk\" (UID: \"e7dd2af0-db13-4782-88d3-4afcadd2202b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk"
Jul 10 02:48:11.533425 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.533388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slvtd\" (UniqueName: \"kubernetes.io/projected/e7dd2af0-db13-4782-88d3-4afcadd2202b-kube-api-access-slvtd\") pod \"limitador-operator-controller-manager-859d7b78bd-h2jwk\" (UID: \"e7dd2af0-db13-4782-88d3-4afcadd2202b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk"
Jul 10 02:48:11.621626 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.621597    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7jxv\" (UniqueName: \"kubernetes.io/projected/9158063b-fe6d-41c8-b8d4-d866cf587a60-kube-api-access-n7jxv\") pod \"9158063b-fe6d-41c8-b8d4-d866cf587a60\" (UID: \"9158063b-fe6d-41c8-b8d4-d866cf587a60\") "
Jul 10 02:48:11.623644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.623620    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9158063b-fe6d-41c8-b8d4-d866cf587a60-kube-api-access-n7jxv" (OuterVolumeSpecName: "kube-api-access-n7jxv") pod "9158063b-fe6d-41c8-b8d4-d866cf587a60" (UID: "9158063b-fe6d-41c8-b8d4-d866cf587a60"). InnerVolumeSpecName "kube-api-access-n7jxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:48:11.644947 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.644904    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk"
Jul 10 02:48:11.723139 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.723090    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7jxv\" (UniqueName: \"kubernetes.io/projected/9158063b-fe6d-41c8-b8d4-d866cf587a60-kube-api-access-n7jxv\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:48:11.806664 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:11.806637    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk"]
Jul 10 02:48:11.808486 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:48:11.808460    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7dd2af0_db13_4782_88d3_4afcadd2202b.slice/crio-d2cbf6cf06ba714193b0af33547d1c43aef7a84fe87852abe60a6562552f19b4 WatchSource:0}: Error finding container d2cbf6cf06ba714193b0af33547d1c43aef7a84fe87852abe60a6562552f19b4: Status 404 returned error can't find the container with id d2cbf6cf06ba714193b0af33547d1c43aef7a84fe87852abe60a6562552f19b4
Jul 10 02:48:12.104809 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:12.104700    2576 generic.go:356] "Generic (PLEG): container finished" podID="9158063b-fe6d-41c8-b8d4-d866cf587a60" containerID="59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5" exitCode=0
Jul 10 02:48:12.104809 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:12.104761    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:48:12.104809 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:12.104802    2576 scope.go:117] "RemoveContainer" containerID="59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5"
Jul 10 02:48:12.106664 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:12.106629    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk" event={"ID":"e7dd2af0-db13-4782-88d3-4afcadd2202b","Type":"ContainerStarted","Data":"86d0ed2fbe4f867726512734290ed62e161c2c0b1ff064278395ed0db1ce517e"}
Jul 10 02:48:12.106800 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:12.106679    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk" event={"ID":"e7dd2af0-db13-4782-88d3-4afcadd2202b","Type":"ContainerStarted","Data":"d2cbf6cf06ba714193b0af33547d1c43aef7a84fe87852abe60a6562552f19b4"}
Jul 10 02:48:12.106800 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:12.106785    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk"
Jul 10 02:48:12.107801 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:48:12.107771    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-znl76\" is forbidden: User \"system:node:ip-10-0-143-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-93.ec2.internal' and this object" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:48:12.110293 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:48:12.110270    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-znl76\" is forbidden: User \"system:node:ip-10-0-143-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-93.ec2.internal' and this object" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:48:12.114632 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:12.114612    2576 scope.go:117] "RemoveContainer" containerID="59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5"
Jul 10 02:48:12.114934 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:48:12.114915    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5\": container with ID starting with 59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5 not found: ID does not exist" containerID="59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5"
Jul 10 02:48:12.115009 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:12.114947    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5"} err="failed to get container status \"59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5\": rpc error: code = NotFound desc = could not find container \"59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5\": container with ID starting with 59c4e9f76e722431f946c9df2652b31fc580d0be54ecf956f35006a2d06fb6f5 not found: ID does not exist"
Jul 10 02:48:12.131414 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:12.131367    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk" podStartSLOduration=1.131351919 podStartE2EDuration="1.131351919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:12.129323142 +0000 UTC m=+501.938588463" watchObservedRunningTime="2026-07-10 02:48:12.131351919 +0000 UTC m=+501.940617303"
Jul 10 02:48:12.131680 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:48:12.131651    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-znl76\" is forbidden: User \"system:node:ip-10-0-143-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-93.ec2.internal' and this object" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znl76"
Jul 10 02:48:12.804496 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:12.804460    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9158063b-fe6d-41c8-b8d4-d866cf587a60" path="/var/lib/kubelet/pods/9158063b-fe6d-41c8-b8d4-d866cf587a60/volumes"
Jul 10 02:48:16.133383 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:16.133340    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t" podUID="2adba223-5fd4-4082-b2de-24db0ae1933a" containerName="manager" containerID="cri-o://a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938" gracePeriod=2
Jul 10 02:48:16.396302 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:16.396238    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:16.398444 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:48:16.398420    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-z8x2t\" is forbidden: User \"system:node:ip-10-0-143-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-93.ec2.internal' and this object" podUID="2adba223-5fd4-4082-b2de-24db0ae1933a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:16.472056 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:16.472014    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2adba223-5fd4-4082-b2de-24db0ae1933a-extensions-socket-volume\") pod \"2adba223-5fd4-4082-b2de-24db0ae1933a\" (UID: \"2adba223-5fd4-4082-b2de-24db0ae1933a\") "
Jul 10 02:48:16.472282 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:16.472258    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2adba223-5fd4-4082-b2de-24db0ae1933a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2adba223-5fd4-4082-b2de-24db0ae1933a" (UID: "2adba223-5fd4-4082-b2de-24db0ae1933a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:16.472356 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:16.472344    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2adba223-5fd4-4082-b2de-24db0ae1933a-extensions-socket-volume\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:48:16.573050 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:16.573007    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqz6b\" (UniqueName: \"kubernetes.io/projected/2adba223-5fd4-4082-b2de-24db0ae1933a-kube-api-access-gqz6b\") pod \"2adba223-5fd4-4082-b2de-24db0ae1933a\" (UID: \"2adba223-5fd4-4082-b2de-24db0ae1933a\") "
Jul 10 02:48:16.575620 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:16.575584    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2adba223-5fd4-4082-b2de-24db0ae1933a-kube-api-access-gqz6b" (OuterVolumeSpecName: "kube-api-access-gqz6b") pod "2adba223-5fd4-4082-b2de-24db0ae1933a" (UID: "2adba223-5fd4-4082-b2de-24db0ae1933a"). InnerVolumeSpecName "kube-api-access-gqz6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:48:16.673861 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:16.673786    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqz6b\" (UniqueName: \"kubernetes.io/projected/2adba223-5fd4-4082-b2de-24db0ae1933a-kube-api-access-gqz6b\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 02:48:16.799607 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:16.799571    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2adba223-5fd4-4082-b2de-24db0ae1933a" path="/var/lib/kubelet/pods/2adba223-5fd4-4082-b2de-24db0ae1933a/volumes"
Jul 10 02:48:17.139177 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:17.139140    2576 generic.go:356] "Generic (PLEG): container finished" podID="2adba223-5fd4-4082-b2de-24db0ae1933a" containerID="a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938" exitCode=2
Jul 10 02:48:17.139619 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:17.139199    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:17.139619 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:17.139209    2576 scope.go:117] "RemoveContainer" containerID="a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938"
Jul 10 02:48:17.143628 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:48:17.143598    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-z8x2t\" is forbidden: User \"system:node:ip-10-0-143-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-93.ec2.internal' and this object" podUID="2adba223-5fd4-4082-b2de-24db0ae1933a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8x2t"
Jul 10 02:48:17.147927 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:17.147909    2576 scope.go:117] "RemoveContainer" containerID="a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938"
Jul 10 02:48:17.148212 ip-10-0-143-93 kubenswrapper[2576]: E0710 02:48:17.148194    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938\": container with ID starting with a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938 not found: ID does not exist" containerID="a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938"
Jul 10 02:48:17.148265 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:17.148222    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938"} err="failed to get container status \"a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938\": rpc error: code = NotFound desc = could not find container \"a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938\": container with ID starting with a5f0f443e1484e59f26bfa1c2011b6b28cd6b26a0587585152e1cda5373b5938 not found: ID does not exist"
Jul 10 02:48:23.117766 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:23.117721    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h2jwk"
Jul 10 02:48:40.957544 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:40.957509    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"]
Jul 10 02:48:40.958119 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:40.958027    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2adba223-5fd4-4082-b2de-24db0ae1933a" containerName="manager"
Jul 10 02:48:40.958119 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:40.958048    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2adba223-5fd4-4082-b2de-24db0ae1933a" containerName="manager"
Jul 10 02:48:40.958207 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:40.958143    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2adba223-5fd4-4082-b2de-24db0ae1933a" containerName="manager"
Jul 10 02:48:40.965512 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:40.965489    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:40.972677 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:40.972654    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7jcvp\""
Jul 10 02:48:41.016580 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.016548    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"]
Jul 10 02:48:41.097611 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.097572    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.097840 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.097706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.097840 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.097803    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.097979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.097845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.097979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.097892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.097979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.097920    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fx8l\" (UniqueName: \"kubernetes.io/projected/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-kube-api-access-5fx8l\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.097979 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.097948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.098142 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.097977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.098142 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.098000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.198701 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.198663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.198911 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.198712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.198911 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.198871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fx8l\" (UniqueName: \"kubernetes.io/projected/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-kube-api-access-5fx8l\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199034 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.198919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199034 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.198952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199034 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.198981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199242 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.199043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199242 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.199181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199242 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.199183    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199408 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.199249    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199408 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.199276    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199408 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.199317    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199408 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.199369    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.199591 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.199464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.201406 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.201376    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.201723 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.201702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.207319 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.207299    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.207715 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.207672    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fx8l\" (UniqueName: \"kubernetes.io/projected/f12035fc-fa7a-4e5b-bb17-870f656e5c2e-kube-api-access-5fx8l\") pod \"maas-default-gateway-openshift-default-56654dcf7-j9rbd\" (UID: \"f12035fc-fa7a-4e5b-bb17-870f656e5c2e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.275805 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.275775    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:41.411901 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.411866    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"]
Jul 10 02:48:41.415802 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:48:41.415775    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12035fc_fa7a_4e5b_bb17_870f656e5c2e.slice/crio-1c7c62172422c7b0d1b60d6f5cace3512611deb26027a2703e24938fe4828722 WatchSource:0}: Error finding container 1c7c62172422c7b0d1b60d6f5cace3512611deb26027a2703e24938fe4828722: Status 404 returned error can't find the container with id 1c7c62172422c7b0d1b60d6f5cace3512611deb26027a2703e24938fe4828722
Jul 10 02:48:41.418213 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.418181    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 02:48:41.418293 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.418244    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 02:48:41.418293 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:41.418273    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 02:48:42.266101 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:42.266015    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd" event={"ID":"f12035fc-fa7a-4e5b-bb17-870f656e5c2e","Type":"ContainerStarted","Data":"9d838d4693c12139cae7b2234402f2541aa4195d7d0b6e0949067112e3b35787"}
Jul 10 02:48:42.266101 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:42.266081    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd" event={"ID":"f12035fc-fa7a-4e5b-bb17-870f656e5c2e","Type":"ContainerStarted","Data":"1c7c62172422c7b0d1b60d6f5cace3512611deb26027a2703e24938fe4828722"}
Jul 10 02:48:42.276012 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:42.275968    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:42.280475 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:42.280450    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:42.289304 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:42.289254    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd" podStartSLOduration=2.289238312 podStartE2EDuration="2.289238312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:42.287008535 +0000 UTC m=+532.096273839" watchObservedRunningTime="2026-07-10 02:48:42.289238312 +0000 UTC m=+532.098503613"
Jul 10 02:48:43.271183 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:43.271154    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:48:43.272198 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:48:43.272180    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:49:15.473977 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.473943    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"]
Jul 10 02:49:15.476857 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.476829    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"
Jul 10 02:49:15.481035 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.481009    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 02:49:15.481290 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.481009    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4qjv5\""
Jul 10 02:49:15.481938 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.481914    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 02:49:15.482038 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.481921    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 10 02:49:15.491404 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.491378    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"]
Jul 10 02:49:15.623119 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.623074    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7229b95d-6f40-47e6-bfbc-a13edde8d16c-data\") pod \"postgres-69fb4594cf-sjrx2\" (UID: \"7229b95d-6f40-47e6-bfbc-a13edde8d16c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"
Jul 10 02:49:15.623302 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.623139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl647\" (UniqueName: \"kubernetes.io/projected/7229b95d-6f40-47e6-bfbc-a13edde8d16c-kube-api-access-nl647\") pod \"postgres-69fb4594cf-sjrx2\" (UID: \"7229b95d-6f40-47e6-bfbc-a13edde8d16c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"
Jul 10 02:49:15.724297 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.724208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7229b95d-6f40-47e6-bfbc-a13edde8d16c-data\") pod \"postgres-69fb4594cf-sjrx2\" (UID: \"7229b95d-6f40-47e6-bfbc-a13edde8d16c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"
Jul 10 02:49:15.724297 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.724271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl647\" (UniqueName: \"kubernetes.io/projected/7229b95d-6f40-47e6-bfbc-a13edde8d16c-kube-api-access-nl647\") pod \"postgres-69fb4594cf-sjrx2\" (UID: \"7229b95d-6f40-47e6-bfbc-a13edde8d16c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"
Jul 10 02:49:15.724644 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.724626    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7229b95d-6f40-47e6-bfbc-a13edde8d16c-data\") pod \"postgres-69fb4594cf-sjrx2\" (UID: \"7229b95d-6f40-47e6-bfbc-a13edde8d16c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"
Jul 10 02:49:15.742857 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.742827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl647\" (UniqueName: \"kubernetes.io/projected/7229b95d-6f40-47e6-bfbc-a13edde8d16c-kube-api-access-nl647\") pod \"postgres-69fb4594cf-sjrx2\" (UID: \"7229b95d-6f40-47e6-bfbc-a13edde8d16c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"
Jul 10 02:49:15.789466 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.789428    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"
Jul 10 02:49:15.929794 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:15.929768    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"]
Jul 10 02:49:15.932485 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:49:15.932457    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7229b95d_6f40_47e6_bfbc_a13edde8d16c.slice/crio-66fc2b8dd19a03fc33aafdae49e6d51a96445fb2f77d1a4bf75975f2426b802a WatchSource:0}: Error finding container 66fc2b8dd19a03fc33aafdae49e6d51a96445fb2f77d1a4bf75975f2426b802a: Status 404 returned error can't find the container with id 66fc2b8dd19a03fc33aafdae49e6d51a96445fb2f77d1a4bf75975f2426b802a
Jul 10 02:49:16.437378 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:16.437336    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2" event={"ID":"7229b95d-6f40-47e6-bfbc-a13edde8d16c","Type":"ContainerStarted","Data":"66fc2b8dd19a03fc33aafdae49e6d51a96445fb2f77d1a4bf75975f2426b802a"}
Jul 10 02:49:21.465020 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:21.464979    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2" event={"ID":"7229b95d-6f40-47e6-bfbc-a13edde8d16c","Type":"ContainerStarted","Data":"bca460df0c7501e67dec369afff002558e5d6f549ef3439a4fbd4e8dc05387bd"}
Jul 10 02:49:21.465490 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:21.465058    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"
Jul 10 02:49:21.482223 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:21.482175    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2" podStartSLOduration=1.828598562 podStartE2EDuration="6.48215932s" totalImagesPullingTime="4.653560758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:49:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:49:21.480317974 +0000 UTC m=+571.289583277" watchObservedRunningTime="2026-07-10 02:49:21.48215932 +0000 UTC m=+571.291424621"
Jul 10 02:49:27.500828 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:27.500798    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjrx2"
Jul 10 02:49:50.608770 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:50.608721    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:49:50.614459 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:49:50.614438    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:50:16.743702 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:16.743666    2576 generic.go:356] "Generic (PLEG): container finished" podID="f12035fc-fa7a-4e5b-bb17-870f656e5c2e" containerID="9d838d4693c12139cae7b2234402f2541aa4195d7d0b6e0949067112e3b35787" exitCode=0
Jul 10 02:50:16.744114 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:16.743738    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd" event={"ID":"f12035fc-fa7a-4e5b-bb17-870f656e5c2e","Type":"ContainerDied","Data":"9d838d4693c12139cae7b2234402f2541aa4195d7d0b6e0949067112e3b35787"}
Jul 10 02:50:16.744208 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:16.744182    2576 scope.go:117] "RemoveContainer" containerID="9d838d4693c12139cae7b2234402f2541aa4195d7d0b6e0949067112e3b35787"
Jul 10 02:50:16.745190 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:16.745166    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 02:50:16.745248 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:16.745235    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 02:50:16.745296 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:16.745272    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 02:50:17.751855 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:17.751821    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd" event={"ID":"f12035fc-fa7a-4e5b-bb17-870f656e5c2e","Type":"ContainerStarted","Data":"931722c5ac432b12e2cb191d49be4c6fb819e29dccca895ba8731422ed3ebd60"}
Jul 10 02:50:18.275992 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:18.275953    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:50:18.276169 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:18.276124    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j9rbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 10 02:50:18.276213 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:18.276176    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd" podUID="f12035fc-fa7a-4e5b-bb17-870f656e5c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 10 02:50:19.276869 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:19.276832    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j9rbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 10 02:50:19.277242 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:19.276916    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd" podUID="f12035fc-fa7a-4e5b-bb17-870f656e5c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 10 02:50:20.276670 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:20.276634    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j9rbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 10 02:50:20.276850 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:20.276687    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd" podUID="f12035fc-fa7a-4e5b-bb17-870f656e5c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 10 02:50:21.276910 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:21.276874    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j9rbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 10 02:50:21.277340 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:21.276927    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd" podUID="f12035fc-fa7a-4e5b-bb17-870f656e5c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 10 02:50:22.279871 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:22.279842    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:50:22.280290 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:22.280127    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:50:22.280879 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:50:22.280855    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j9rbd"
Jul 10 02:51:56.868948 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.868865    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"]
Jul 10 02:51:56.872861 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.872837    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:56.875408 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.875387    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 02:51:56.876426 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.876395    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4zkwr\""
Jul 10 02:51:56.876546 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.876452    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 02:51:56.876546 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.876498    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 10 02:51:56.882678 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.882654    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"]
Jul 10 02:51:56.951960 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.951918    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:56.952142 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.952004    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78b7dd38-5b46-4eda-9aa8-660669952819-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:56.952142 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.952092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:56.952142 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.952124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xd9c\" (UniqueName: \"kubernetes.io/projected/78b7dd38-5b46-4eda-9aa8-660669952819-kube-api-access-5xd9c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:56.952277 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.952145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:56.952277 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:56.952169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.052843 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.052801    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78b7dd38-5b46-4eda-9aa8-660669952819-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.053037 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.052883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.053037 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.052908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xd9c\" (UniqueName: \"kubernetes.io/projected/78b7dd38-5b46-4eda-9aa8-660669952819-kube-api-access-5xd9c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.053037 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.052934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.053037 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.052965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.053232 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.053038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.053370 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.053349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.053370 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.053363    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.053496 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.053461    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.055207 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.055177    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78b7dd38-5b46-4eda-9aa8-660669952819-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.055408 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.055385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78b7dd38-5b46-4eda-9aa8-660669952819-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.061376 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.061352    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xd9c\" (UniqueName: \"kubernetes.io/projected/78b7dd38-5b46-4eda-9aa8-660669952819-kube-api-access-5xd9c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr\" (UID: \"78b7dd38-5b46-4eda-9aa8-660669952819\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.185771 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.185653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:51:57.322430 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.322407    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"]
Jul 10 02:51:57.324569 ip-10-0-143-93 kubenswrapper[2576]: W0710 02:51:57.324542    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78b7dd38_5b46_4eda_9aa8_660669952819.slice/crio-122c62788a4d77e5abe2f34a125cf61ce85eaa9fb14177c25cbf36837244d281 WatchSource:0}: Error finding container 122c62788a4d77e5abe2f34a125cf61ce85eaa9fb14177c25cbf36837244d281: Status 404 returned error can't find the container with id 122c62788a4d77e5abe2f34a125cf61ce85eaa9fb14177c25cbf36837244d281
Jul 10 02:51:57.325886 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:57.325867    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:51:58.254888 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:58.254855    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr" event={"ID":"78b7dd38-5b46-4eda-9aa8-660669952819","Type":"ContainerStarted","Data":"122c62788a4d77e5abe2f34a125cf61ce85eaa9fb14177c25cbf36837244d281"}
Jul 10 02:51:59.260853 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:59.260816    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr" event={"ID":"78b7dd38-5b46-4eda-9aa8-660669952819","Type":"ContainerStarted","Data":"9707976e451bd8ef1e5234a458c138115061d76677a537889c9f216b1eb1360d"}
Jul 10 02:51:59.280519 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:51:59.280458    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr" podStartSLOduration=2.27143693 podStartE2EDuration="3.28044266s" totalImagesPullingTime="1.00900573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:51:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:51:59.278182091 +0000 UTC m=+729.087447393" watchObservedRunningTime="2026-07-10 02:51:59.28044266 +0000 UTC m=+729.089707962"
Jul 10 02:52:07.186813 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:52:07.186774    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:52:07.187275 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:52:07.186838    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:52:07.199387 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:52:07.199363    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:52:07.311813 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:52:07.311784    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr"
Jul 10 02:54:50.643495 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:54:50.643419    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:54:50.645491 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:54:50.645464    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:59:50.680647 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:59:50.680564    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 02:59:50.681187 ip-10-0-143-93 kubenswrapper[2576]: I0710 02:59:50.680897    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:00:00.221722 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:00:00.221686    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7"]
Jul 10 03:00:00.225172 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:00:00.225155    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7"
Jul 10 03:00:00.243563 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:00:00.243537    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nhxqz\""
Jul 10 03:00:00.245613 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:00:00.245595    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7"
Jul 10 03:00:00.292384 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:00:00.292350    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7"]
Jul 10 03:00:00.391708 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:00:00.391682    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7"]
Jul 10 03:00:00.393700 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:00:00.393668    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a016a8_5b5b_4fbf_aa5d_3226c9fbd253.slice/crio-39c5cc1a40a8e445041412363757bfc1c40b593f534bc57a3daa22e113e4170f WatchSource:0}: Error finding container 39c5cc1a40a8e445041412363757bfc1c40b593f534bc57a3daa22e113e4170f: Status 404 returned error can't find the container with id 39c5cc1a40a8e445041412363757bfc1c40b593f534bc57a3daa22e113e4170f
Jul 10 03:00:00.395792 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:00:00.395775    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 03:00:00.664329 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:00:00.664231    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7" event={"ID":"68a016a8-5b5b-4fbf-aa5d-3226c9fbd253","Type":"ContainerStarted","Data":"39c5cc1a40a8e445041412363757bfc1c40b593f534bc57a3daa22e113e4170f"}
Jul 10 03:00:04.685473 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:00:04.685429    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7" event={"ID":"68a016a8-5b5b-4fbf-aa5d-3226c9fbd253","Type":"ContainerStarted","Data":"aa59c8aa64d2a701d774a3aea7312172121c9f13fd7e679f369502cb4af41335"}
Jul 10 03:00:04.706840 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:00:04.706781    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7" podStartSLOduration=1.5563159020000001 podStartE2EDuration="4.706760128s" totalImagesPullingTime="3.150444226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:00:04.704124243 +0000 UTC m=+1214.513389559" watchObservedRunningTime="2026-07-10 03:00:04.706760128 +0000 UTC m=+1214.516025422"
Jul 10 03:01:00.012594 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:00.012505    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7"]
Jul 10 03:01:00.015356 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:00.012795    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7" podUID="68a016a8-5b5b-4fbf-aa5d-3226c9fbd253" containerName="cleanup" containerID="cri-o://aa59c8aa64d2a701d774a3aea7312172121c9f13fd7e679f369502cb4af41335" gracePeriod=30
Jul 10 03:01:30.112598 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:30.112555    2576 generic.go:356] "Generic (PLEG): container finished" podID="68a016a8-5b5b-4fbf-aa5d-3226c9fbd253" containerID="aa59c8aa64d2a701d774a3aea7312172121c9f13fd7e679f369502cb4af41335" exitCode=137
Jul 10 03:01:30.113102 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:30.112626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7" event={"ID":"68a016a8-5b5b-4fbf-aa5d-3226c9fbd253","Type":"ContainerDied","Data":"aa59c8aa64d2a701d774a3aea7312172121c9f13fd7e679f369502cb4af41335"}
Jul 10 03:01:30.207017 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:30.206992    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7"
Jul 10 03:01:31.118539 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:31.118447    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7" event={"ID":"68a016a8-5b5b-4fbf-aa5d-3226c9fbd253","Type":"ContainerDied","Data":"39c5cc1a40a8e445041412363757bfc1c40b593f534bc57a3daa22e113e4170f"}
Jul 10 03:01:31.119001 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:31.118545    2576 scope.go:117] "RemoveContainer" containerID="aa59c8aa64d2a701d774a3aea7312172121c9f13fd7e679f369502cb4af41335"
Jul 10 03:01:31.119001 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:31.118463    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7"
Jul 10 03:01:31.135065 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:31.135042    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7"]
Jul 10 03:01:31.139097 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:31.139074    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-rptg7"]
Jul 10 03:01:32.798548 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:01:32.798512    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68a016a8-5b5b-4fbf-aa5d-3226c9fbd253" path="/var/lib/kubelet/pods/68a016a8-5b5b-4fbf-aa5d-3226c9fbd253/volumes"
Jul 10 03:04:50.714278 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:04:50.714246    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:04:50.716807 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:04:50.716785    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:09:50.751192 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:09:50.751161    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:09:50.754760 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:09:50.754732    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:10:31.109513 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.109473    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"]
Jul 10 03:10:31.110191 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.110170    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68a016a8-5b5b-4fbf-aa5d-3226c9fbd253" containerName="cleanup"
Jul 10 03:10:31.110242 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.110196    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="68a016a8-5b5b-4fbf-aa5d-3226c9fbd253" containerName="cleanup"
Jul 10 03:10:31.110306 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.110294    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="68a016a8-5b5b-4fbf-aa5d-3226c9fbd253" containerName="cleanup"
Jul 10 03:10:31.113951 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.113930    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.116494 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.116475    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a9b31aac-openshift-default-dockercfg-vvr5c\""
Jul 10 03:10:31.134818 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.134789    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"]
Jul 10 03:10:31.245489 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.245448    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-podinfo\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.245665 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.245508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-credential-socket\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.245665 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.245528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3579e73a-6f73-47e6-8034-ae69a943cf96-istiod-ca-cert\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.245665 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.245585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-socket\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.245847 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.245672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hht2v\" (UniqueName: \"kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-kube-api-access-hht2v\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.245847 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.245698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-envoy\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.245847 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.245718    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-token\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.245847 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.245825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-certs\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.246017 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.245865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-data\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.346473 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346431    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-token\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.346473 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-certs\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.346728 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346493    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-data\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.346728 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346530    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-podinfo\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.346728 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-credential-socket\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.346728 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3579e73a-6f73-47e6-8034-ae69a943cf96-istiod-ca-cert\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.346728 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346673    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-socket\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.347011 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hht2v\" (UniqueName: \"kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-kube-api-access-hht2v\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.347011 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-envoy\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.347011 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346907    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-certs\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.347011 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.346959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-data\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.347163 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.347094    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-socket\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.347163 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.347152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-credential-socket\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.347346 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.347328    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3579e73a-6f73-47e6-8034-ae69a943cf96-istiod-ca-cert\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.349147 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.349126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-envoy\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.349331 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.349310    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-podinfo\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.355116 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.355096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-token\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.355224 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.355203    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hht2v\" (UniqueName: \"kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-kube-api-access-hht2v\") pod \"e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") " pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.428389 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.428309    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:31.568432 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.568396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"]
Jul 10 03:10:31.570018 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:10:31.569986    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3579e73a_6f73_47e6_8034_ae69a943cf96.slice/crio-06f290c7770e8d14cb79b74f98b71474c934618a03cb395acb8c8d97ae34c6ca WatchSource:0}: Error finding container 06f290c7770e8d14cb79b74f98b71474c934618a03cb395acb8c8d97ae34c6ca: Status 404 returned error can't find the container with id 06f290c7770e8d14cb79b74f98b71474c934618a03cb395acb8c8d97ae34c6ca
Jul 10 03:10:31.572159 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.572127    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:10:31.572229 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.572187    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:10:31.572229 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.572217    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:10:31.812410 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.812374    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" event={"ID":"3579e73a-6f73-47e6-8034-ae69a943cf96","Type":"ContainerStarted","Data":"55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797"}
Jul 10 03:10:31.812593 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.812442    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" event={"ID":"3579e73a-6f73-47e6-8034-ae69a943cf96","Type":"ContainerStarted","Data":"06f290c7770e8d14cb79b74f98b71474c934618a03cb395acb8c8d97ae34c6ca"}
Jul 10 03:10:31.832836 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:31.832782    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" podStartSLOduration=0.832765509 podStartE2EDuration="832.765509ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:10:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:10:31.829907206 +0000 UTC m=+1841.639172547" watchObservedRunningTime="2026-07-10 03:10:31.832765509 +0000 UTC m=+1841.642030805"
Jul 10 03:10:32.429173 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:32.429131    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:32.433987 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:32.433960    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:32.817120 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:32.817088    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:32.818164 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:32.818144    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:43.879353 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:43.879313    2576 generic.go:356] "Generic (PLEG): container finished" podID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerID="55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797" exitCode=0
Jul 10 03:10:43.879805 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:43.879357    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" event={"ID":"3579e73a-6f73-47e6-8034-ae69a943cf96","Type":"ContainerDied","Data":"55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797"}
Jul 10 03:10:43.879866 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:43.879840    2576 scope.go:117] "RemoveContainer" containerID="55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797"
Jul 10 03:10:43.880922 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:43.880894    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:10:43.880979 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:43.880960    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:10:43.881016 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:43.880999    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:10:44.886388 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:44.886349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" event={"ID":"3579e73a-6f73-47e6-8034-ae69a943cf96","Type":"ContainerStarted","Data":"09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31"}
Jul 10 03:10:45.429123 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:45.429084    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:45.430313 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:45.430273    2576 patch_prober.go:28] interesting pod/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 10 03:10:45.430448 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:45.430357    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 10 03:10:46.429193 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:46.429138    2576 patch_prober.go:28] interesting pod/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 10 03:10:46.429720 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:46.429255    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 10 03:10:47.429073 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:47.429030    2576 patch_prober.go:28] interesting pod/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 10 03:10:47.429282 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:47.429100    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 10 03:10:47.817182 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:47.817136    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:48.429784 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:48.429731    2576 patch_prober.go:28] interesting pod/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 10 03:10:48.430351 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:48.430318    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 10 03:10:49.432974 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:49.432940    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:10:49.433896 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:10:49.433871    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:11:37.571784 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:37.571680    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"]
Jul 10 03:11:37.572327 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:37.571905    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy" containerID="cri-o://09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31" gracePeriod=30
Jul 10 03:11:42.820670 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.820637    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:11:42.977168 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977134    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-envoy\") pod \"3579e73a-6f73-47e6-8034-ae69a943cf96\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") "
Jul 10 03:11:42.977359 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977177    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-certs\") pod \"3579e73a-6f73-47e6-8034-ae69a943cf96\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") "
Jul 10 03:11:42.977359 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977209    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-credential-socket\") pod \"3579e73a-6f73-47e6-8034-ae69a943cf96\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") "
Jul 10 03:11:42.977359 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977255    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hht2v\" (UniqueName: \"kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-kube-api-access-hht2v\") pod \"3579e73a-6f73-47e6-8034-ae69a943cf96\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") "
Jul 10 03:11:42.977359 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977280    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-podinfo\") pod \"3579e73a-6f73-47e6-8034-ae69a943cf96\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") "
Jul 10 03:11:42.977359 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977351    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-token\") pod \"3579e73a-6f73-47e6-8034-ae69a943cf96\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") "
Jul 10 03:11:42.977622 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977388    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-data\") pod \"3579e73a-6f73-47e6-8034-ae69a943cf96\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") "
Jul 10 03:11:42.977622 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977431    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-socket\") pod \"3579e73a-6f73-47e6-8034-ae69a943cf96\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") "
Jul 10 03:11:42.977622 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977487    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3579e73a-6f73-47e6-8034-ae69a943cf96-istiod-ca-cert\") pod \"3579e73a-6f73-47e6-8034-ae69a943cf96\" (UID: \"3579e73a-6f73-47e6-8034-ae69a943cf96\") "
Jul 10 03:11:42.977622 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977507    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3579e73a-6f73-47e6-8034-ae69a943cf96" (UID: "3579e73a-6f73-47e6-8034-ae69a943cf96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:11:42.977859 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977625    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3579e73a-6f73-47e6-8034-ae69a943cf96" (UID: "3579e73a-6f73-47e6-8034-ae69a943cf96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:11:42.977927 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977909    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:11:42.977987 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977933    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:11:42.978041 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.977980    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3579e73a-6f73-47e6-8034-ae69a943cf96-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3579e73a-6f73-47e6-8034-ae69a943cf96" (UID: "3579e73a-6f73-47e6-8034-ae69a943cf96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:11:42.978095 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.978082    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3579e73a-6f73-47e6-8034-ae69a943cf96" (UID: "3579e73a-6f73-47e6-8034-ae69a943cf96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:11:42.978307 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.978276    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-data" (OuterVolumeSpecName: "istio-data") pod "3579e73a-6f73-47e6-8034-ae69a943cf96" (UID: "3579e73a-6f73-47e6-8034-ae69a943cf96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:11:42.979697 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.979662    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3579e73a-6f73-47e6-8034-ae69a943cf96" (UID: "3579e73a-6f73-47e6-8034-ae69a943cf96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:11:42.979835 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.979709    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3579e73a-6f73-47e6-8034-ae69a943cf96" (UID: "3579e73a-6f73-47e6-8034-ae69a943cf96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:11:42.979915 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.979891    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-kube-api-access-hht2v" (OuterVolumeSpecName: "kube-api-access-hht2v") pod "3579e73a-6f73-47e6-8034-ae69a943cf96" (UID: "3579e73a-6f73-47e6-8034-ae69a943cf96"). InnerVolumeSpecName "kube-api-access-hht2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:11:42.979963 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:42.979924    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-token" (OuterVolumeSpecName: "istio-token") pod "3579e73a-6f73-47e6-8034-ae69a943cf96" (UID: "3579e73a-6f73-47e6-8034-ae69a943cf96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:11:43.078518 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.078470    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3579e73a-6f73-47e6-8034-ae69a943cf96-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:11:43.078518 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.078510    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:11:43.078518 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.078519    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hht2v\" (UniqueName: \"kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-kube-api-access-hht2v\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:11:43.078518 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.078529    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:11:43.078834 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.078538    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:11:43.078834 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.078545    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:11:43.078834 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.078552    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3579e73a-6f73-47e6-8034-ae69a943cf96-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:11:43.180976 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.180887    2576 generic.go:356] "Generic (PLEG): container finished" podID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerID="09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31" exitCode=0
Jul 10 03:11:43.180976 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.180938    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"
Jul 10 03:11:43.180976 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.180949    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" event={"ID":"3579e73a-6f73-47e6-8034-ae69a943cf96","Type":"ContainerDied","Data":"09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31"}
Jul 10 03:11:43.181199 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.180993    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x" event={"ID":"3579e73a-6f73-47e6-8034-ae69a943cf96","Type":"ContainerDied","Data":"06f290c7770e8d14cb79b74f98b71474c934618a03cb395acb8c8d97ae34c6ca"}
Jul 10 03:11:43.181199 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.181014    2576 scope.go:117] "RemoveContainer" containerID="09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31"
Jul 10 03:11:43.191177 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.191159    2576 scope.go:117] "RemoveContainer" containerID="55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797"
Jul 10 03:11:43.200016 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.199933    2576 scope.go:117] "RemoveContainer" containerID="09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31"
Jul 10 03:11:43.200284 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:11:43.200266    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31\": container with ID starting with 09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31 not found: ID does not exist" containerID="09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31"
Jul 10 03:11:43.200358 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.200297    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31"} err="failed to get container status \"09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31\": rpc error: code = NotFound desc = could not find container \"09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31\": container with ID starting with 09510f1be3cc1b8d5112ed8c7f8ca9d765ef095e86ffe447a047a1e5e9c06b31 not found: ID does not exist"
Jul 10 03:11:43.200358 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.200322    2576 scope.go:117] "RemoveContainer" containerID="55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797"
Jul 10 03:11:43.200581 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:11:43.200562    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797\": container with ID starting with 55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797 not found: ID does not exist" containerID="55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797"
Jul 10 03:11:43.200627 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.200586    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797"} err="failed to get container status \"55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797\": rpc error: code = NotFound desc = could not find container \"55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797\": container with ID starting with 55ed42d1bb04d4ccc77867831b9c853008ccf16d69d2371020f17e579e5f1797 not found: ID does not exist"
Jul 10 03:11:43.201894 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.201873    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"]
Jul 10 03:11:43.208567 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:43.208542    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a9b31aac-openshift-default-b8bbb768f-7256x"]
Jul 10 03:11:44.799785 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:44.799735    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" path="/var/lib/kubelet/pods/3579e73a-6f73-47e6-8034-ae69a943cf96/volumes"
Jul 10 03:11:49.464129 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.464096    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"]
Jul 10 03:11:49.464708 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.464687    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy"
Jul 10 03:11:49.464823 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.464711    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy"
Jul 10 03:11:49.464884 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.464868    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy"
Jul 10 03:11:49.464932 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.464885    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy"
Jul 10 03:11:49.469700 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.469674    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.474890 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.474868    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c9655e49-openshift-default-dockercfg-2sd5w\""
Jul 10 03:11:49.482247 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.482218    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"]
Jul 10 03:11:49.537810 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.537778    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68609a3d-24a1-41aa-a194-889734c45276-istiod-ca-cert\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.537810 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.537814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-socket\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.538017 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.537838    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-credential-socket\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.538017 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.537898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-data\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.538017 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.537949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68609a3d-24a1-41aa-a194-889734c45276-istio-podinfo\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.538017 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.537968    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-istio-token\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.538017 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.537989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-envoy\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.538190 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.538058    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-certs\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.538190 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.538135    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4fzr\" (UniqueName: \"kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-kube-api-access-j4fzr\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.638940 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.638891    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-istio-token\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.638940 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.638943    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-envoy\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639213 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639002    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-certs\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639213 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4fzr\" (UniqueName: \"kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-kube-api-access-j4fzr\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639213 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68609a3d-24a1-41aa-a194-889734c45276-istiod-ca-cert\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639213 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-socket\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639213 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-credential-socket\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639499 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639228    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-data\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639499 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68609a3d-24a1-41aa-a194-889734c45276-istio-podinfo\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639499 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639482    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-certs\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639656 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639566    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-socket\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639656 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639591    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-credential-socket\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639799 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-data\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.639925 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.639901    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68609a3d-24a1-41aa-a194-889734c45276-istiod-ca-cert\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.641421 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.641399    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-envoy\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.641842 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.641819    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68609a3d-24a1-41aa-a194-889734c45276-istio-podinfo\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.648627 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.648605    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-istio-token\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.648707 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.648687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4fzr\" (UniqueName: \"kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-kube-api-access-j4fzr\") pod \"e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") " pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.781674 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.781580    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:49.919943 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.919917    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"]
Jul 10 03:11:49.921761 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:11:49.921723    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68609a3d_24a1_41aa_a194_889734c45276.slice/crio-abb3d3147d5443d128cf35ea454ae0c5d3848377f71f46eaa6f81e4302df06f8 WatchSource:0}: Error finding container abb3d3147d5443d128cf35ea454ae0c5d3848377f71f46eaa6f81e4302df06f8: Status 404 returned error can't find the container with id abb3d3147d5443d128cf35ea454ae0c5d3848377f71f46eaa6f81e4302df06f8
Jul 10 03:11:49.924031 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.924000    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:11:49.924098 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.924069    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:11:49.924138 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:49.924095    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:11:50.220692 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:50.220657    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" event={"ID":"68609a3d-24a1-41aa-a194-889734c45276","Type":"ContainerStarted","Data":"45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df"}
Jul 10 03:11:50.220903 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:50.220717    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" event={"ID":"68609a3d-24a1-41aa-a194-889734c45276","Type":"ContainerStarted","Data":"abb3d3147d5443d128cf35ea454ae0c5d3848377f71f46eaa6f81e4302df06f8"}
Jul 10 03:11:50.242515 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:50.242462    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" podStartSLOduration=1.242446574 podStartE2EDuration="1.242446574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:11:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:11:50.23996907 +0000 UTC m=+1920.049234372" watchObservedRunningTime="2026-07-10 03:11:50.242446574 +0000 UTC m=+1920.051711876"
Jul 10 03:11:50.782635 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:50.782601    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:50.787630 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:50.787598    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:51.229440 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:51.228304    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:11:51.229440 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:11:51.229330    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:12:02.286347 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:02.286312    2576 generic.go:356] "Generic (PLEG): container finished" podID="68609a3d-24a1-41aa-a194-889734c45276" containerID="45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df" exitCode=0
Jul 10 03:12:02.286801 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:02.286385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" event={"ID":"68609a3d-24a1-41aa-a194-889734c45276","Type":"ContainerDied","Data":"45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df"}
Jul 10 03:12:02.286854 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:02.286840    2576 scope.go:117] "RemoveContainer" containerID="45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df"
Jul 10 03:12:02.287908 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:02.287881    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:12:02.287973 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:02.287960    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:12:02.288017 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:02.288007    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:12:03.294169 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:03.294135    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" event={"ID":"68609a3d-24a1-41aa-a194-889734c45276","Type":"ContainerStarted","Data":"2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1"}
Jul 10 03:12:03.782637 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:03.782593    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:12:03.783739 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:03.783714    2576 patch_prober.go:28] interesting pod/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 10 03:12:03.783807 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:03.783788    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 10 03:12:04.782412 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:04.782374    2576 patch_prober.go:28] interesting pod/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 10 03:12:04.782871 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:04.782430    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 10 03:12:05.782802 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:05.782738    2576 patch_prober.go:28] interesting pod/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 10 03:12:05.783209 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:05.782836    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 10 03:12:06.228269 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:06.228229    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:12:06.781954 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:06.781918    2576 patch_prober.go:28] interesting pod/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 10 03:12:06.782131 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:06.781980    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 10 03:12:07.786403 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:07.786370    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:12:07.787486 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:12:07.787462    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:13:56.066428 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:13:56.066391    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"]
Jul 10 03:13:56.067033 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:13:56.066603    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy" containerID="cri-o://2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1" gracePeriod=30
Jul 10 03:14:01.330415 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.330349    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:14:01.428133 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428097    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-envoy\") pod \"68609a3d-24a1-41aa-a194-889734c45276\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") "
Jul 10 03:14:01.428133 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428134    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-certs\") pod \"68609a3d-24a1-41aa-a194-889734c45276\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") "
Jul 10 03:14:01.428458 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428156    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-istio-token\") pod \"68609a3d-24a1-41aa-a194-889734c45276\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") "
Jul 10 03:14:01.428458 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428180    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-socket\") pod \"68609a3d-24a1-41aa-a194-889734c45276\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") "
Jul 10 03:14:01.428458 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428221    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4fzr\" (UniqueName: \"kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-kube-api-access-j4fzr\") pod \"68609a3d-24a1-41aa-a194-889734c45276\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") "
Jul 10 03:14:01.428458 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428385    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-data\") pod \"68609a3d-24a1-41aa-a194-889734c45276\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") "
Jul 10 03:14:01.428458 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428420    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68609a3d-24a1-41aa-a194-889734c45276-istiod-ca-cert\") pod \"68609a3d-24a1-41aa-a194-889734c45276\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") "
Jul 10 03:14:01.428739 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428461    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "68609a3d-24a1-41aa-a194-889734c45276" (UID: "68609a3d-24a1-41aa-a194-889734c45276"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:14:01.428739 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428516    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "68609a3d-24a1-41aa-a194-889734c45276" (UID: "68609a3d-24a1-41aa-a194-889734c45276"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:14:01.428739 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428549    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68609a3d-24a1-41aa-a194-889734c45276-istio-podinfo\") pod \"68609a3d-24a1-41aa-a194-889734c45276\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") "
Jul 10 03:14:01.428739 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428594    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-credential-socket\") pod \"68609a3d-24a1-41aa-a194-889734c45276\" (UID: \"68609a3d-24a1-41aa-a194-889734c45276\") "
Jul 10 03:14:01.428984 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428885    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68609a3d-24a1-41aa-a194-889734c45276-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "68609a3d-24a1-41aa-a194-889734c45276" (UID: "68609a3d-24a1-41aa-a194-889734c45276"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:14:01.429054 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.428986    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68609a3d-24a1-41aa-a194-889734c45276-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:01.429054 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.429006    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:01.429054 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.429019    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:01.429208 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.429127    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "68609a3d-24a1-41aa-a194-889734c45276" (UID: "68609a3d-24a1-41aa-a194-889734c45276"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:14:01.429284 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.429260    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-data" (OuterVolumeSpecName: "istio-data") pod "68609a3d-24a1-41aa-a194-889734c45276" (UID: "68609a3d-24a1-41aa-a194-889734c45276"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:14:01.430591 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.430550    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "68609a3d-24a1-41aa-a194-889734c45276" (UID: "68609a3d-24a1-41aa-a194-889734c45276"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:14:01.430926 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.430902    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-istio-token" (OuterVolumeSpecName: "istio-token") pod "68609a3d-24a1-41aa-a194-889734c45276" (UID: "68609a3d-24a1-41aa-a194-889734c45276"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:14:01.431054 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.431033    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68609a3d-24a1-41aa-a194-889734c45276-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "68609a3d-24a1-41aa-a194-889734c45276" (UID: "68609a3d-24a1-41aa-a194-889734c45276"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:14:01.431100 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.431068    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-kube-api-access-j4fzr" (OuterVolumeSpecName: "kube-api-access-j4fzr") pod "68609a3d-24a1-41aa-a194-889734c45276" (UID: "68609a3d-24a1-41aa-a194-889734c45276"). InnerVolumeSpecName "kube-api-access-j4fzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:14:01.529877 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.529840    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68609a3d-24a1-41aa-a194-889734c45276-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:01.529877 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.529871    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:01.529877 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.529881    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:01.530104 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.529891    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:01.530104 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.529900    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4fzr\" (UniqueName: \"kubernetes.io/projected/68609a3d-24a1-41aa-a194-889734c45276-kube-api-access-j4fzr\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:01.530104 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.529908    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68609a3d-24a1-41aa-a194-889734c45276-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:01.894232 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.894195    2576 generic.go:356] "Generic (PLEG): container finished" podID="68609a3d-24a1-41aa-a194-889734c45276" containerID="2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1" exitCode=0
Jul 10 03:14:01.894415 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.894248    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"
Jul 10 03:14:01.894415 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.894288    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" event={"ID":"68609a3d-24a1-41aa-a194-889734c45276","Type":"ContainerDied","Data":"2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1"}
Jul 10 03:14:01.894415 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.894346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf" event={"ID":"68609a3d-24a1-41aa-a194-889734c45276","Type":"ContainerDied","Data":"abb3d3147d5443d128cf35ea454ae0c5d3848377f71f46eaa6f81e4302df06f8"}
Jul 10 03:14:01.894415 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.894371    2576 scope.go:117] "RemoveContainer" containerID="2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1"
Jul 10 03:14:01.904005 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.903987    2576 scope.go:117] "RemoveContainer" containerID="45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df"
Jul 10 03:14:01.912466 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.912434    2576 scope.go:117] "RemoveContainer" containerID="2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1"
Jul 10 03:14:01.912725 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:14:01.912706    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1\": container with ID starting with 2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1 not found: ID does not exist" containerID="2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1"
Jul 10 03:14:01.912815 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.912737    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1"} err="failed to get container status \"2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1\": rpc error: code = NotFound desc = could not find container \"2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1\": container with ID starting with 2eb2679b38f82602d36f4a4086710395572e395f2957452e4ee336c173f819d1 not found: ID does not exist"
Jul 10 03:14:01.912815 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.912778    2576 scope.go:117] "RemoveContainer" containerID="45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df"
Jul 10 03:14:01.913082 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:14:01.913061    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df\": container with ID starting with 45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df not found: ID does not exist" containerID="45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df"
Jul 10 03:14:01.913124 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.913089    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df"} err="failed to get container status \"45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df\": rpc error: code = NotFound desc = could not find container \"45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df\": container with ID starting with 45ef803ca2e402d9c5c8613d345658cf284b6eab091323770d2789366bef32df not found: ID does not exist"
Jul 10 03:14:01.917012 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.916989    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"]
Jul 10 03:14:01.921544 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:01.921521    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c9655e49-openshift-default-66db77978c-bjpsf"]
Jul 10 03:14:02.798988 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:02.798953    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68609a3d-24a1-41aa-a194-889734c45276" path="/var/lib/kubelet/pods/68609a3d-24a1-41aa-a194-889734c45276/volumes"
Jul 10 03:14:08.000677 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.000648    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"]
Jul 10 03:14:08.001251 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.001165    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy"
Jul 10 03:14:08.001251 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.001180    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3579e73a-6f73-47e6-8034-ae69a943cf96" containerName="istio-proxy"
Jul 10 03:14:08.001251 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.001189    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy"
Jul 10 03:14:08.001251 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.001195    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy"
Jul 10 03:14:08.001251 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.001207    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy"
Jul 10 03:14:08.001251 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.001212    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy"
Jul 10 03:14:08.001463 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.001278    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy"
Jul 10 03:14:08.005857 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.005836    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.008717 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.008696    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-5374048d-openshift-default-dockercfg-t7sgp\""
Jul 10 03:14:08.034320 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.034288    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"]
Jul 10 03:14:08.094554 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.094514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-podinfo\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.094701 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.094570    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-data\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.094701 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.094607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmhj4\" (UniqueName: \"kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-kube-api-access-lmhj4\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.094701 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.094635    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-certs\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.094701 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.094663    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istiod-ca-cert\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.094871 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.094722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-credential-socket\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.094910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.094876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-token\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.094944 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.094931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-envoy\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.095007 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.094988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-socket\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.196419 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-token\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.196419 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196426    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-envoy\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.196675 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-socket\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.196675 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-podinfo\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.196675 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196535    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-data\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.196675 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmhj4\" (UniqueName: \"kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-kube-api-access-lmhj4\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.196675 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-certs\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.196675 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196624    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istiod-ca-cert\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.196675 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196656    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-credential-socket\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.197092 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-socket\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.197092 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.196992    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-data\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.197194 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.197095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-certs\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.197262 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.197238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-credential-socket\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.197436 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.197417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istiod-ca-cert\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.199036 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.199009    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-envoy\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.199144 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.199115    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-podinfo\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.207412 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.207391    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmhj4\" (UniqueName: \"kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-kube-api-access-lmhj4\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.208042 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.208012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-token\") pod \"e2e-derive-5374048d-openshift-default-855cb98967-hv747\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") " pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.319997 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.319898    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:08.465244 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.465204    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"]
Jul 10 03:14:08.469210 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:14:08.469185    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe7c1d18_2aaa_4338_91f1_9372ddfde274.slice/crio-670e39206d55ea4393f5984e6a36427333f60eefbcb1c81b196755a3df267fc0 WatchSource:0}: Error finding container 670e39206d55ea4393f5984e6a36427333f60eefbcb1c81b196755a3df267fc0: Status 404 returned error can't find the container with id 670e39206d55ea4393f5984e6a36427333f60eefbcb1c81b196755a3df267fc0
Jul 10 03:14:08.471343 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.471310    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:14:08.471405 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.471378    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:14:08.471459 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.471406    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:14:08.932626 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.932588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747" event={"ID":"fe7c1d18-2aaa-4338-91f1-9372ddfde274","Type":"ContainerStarted","Data":"7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b"}
Jul 10 03:14:08.932851 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.932647    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747" event={"ID":"fe7c1d18-2aaa-4338-91f1-9372ddfde274","Type":"ContainerStarted","Data":"670e39206d55ea4393f5984e6a36427333f60eefbcb1c81b196755a3df267fc0"}
Jul 10 03:14:08.954038 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:08.953902    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747" podStartSLOduration=1.9538809480000001 podStartE2EDuration="1.953880948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:14:08.953728189 +0000 UTC m=+2058.762993492" watchObservedRunningTime="2026-07-10 03:14:08.953880948 +0000 UTC m=+2058.763146267"
Jul 10 03:14:09.320319 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:09.320281    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:09.325193 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:09.325168    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:09.937880 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:09.937847    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:09.938791 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:09.938769    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:21.972759 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:21.972710    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"]
Jul 10 03:14:21.973147 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:21.972941    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747" podUID="fe7c1d18-2aaa-4338-91f1-9372ddfde274" containerName="istio-proxy" containerID="cri-o://7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b" gracePeriod=30
Jul 10 03:14:24.940504 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:24.940463    2576 patch_prober.go:28] interesting pod/e2e-derive-5374048d-openshift-default-855cb98967-hv747 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 03:14:24.940910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:24.940531    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747" podUID="fe7c1d18-2aaa-4338-91f1-9372ddfde274" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 03:14:27.224991 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.224960    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:27.283790 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.283685    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-envoy\") pod \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") "
Jul 10 03:14:27.283790 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.283786    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istiod-ca-cert\") pod \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") "
Jul 10 03:14:27.284015 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.283815    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-certs\") pod \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") "
Jul 10 03:14:27.284015 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.283851    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-socket\") pod \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") "
Jul 10 03:14:27.284015 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.283880    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmhj4\" (UniqueName: \"kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-kube-api-access-lmhj4\") pod \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") "
Jul 10 03:14:27.284015 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.283926    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-podinfo\") pod \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") "
Jul 10 03:14:27.284015 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.283949    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-data\") pod \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") "
Jul 10 03:14:27.284287 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284076    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-credential-socket\") pod \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") "
Jul 10 03:14:27.284287 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284098    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-token\") pod \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\" (UID: \"fe7c1d18-2aaa-4338-91f1-9372ddfde274\") "
Jul 10 03:14:27.284287 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284104    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe7c1d18-2aaa-4338-91f1-9372ddfde274" (UID: "fe7c1d18-2aaa-4338-91f1-9372ddfde274"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:14:27.284287 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284189    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe7c1d18-2aaa-4338-91f1-9372ddfde274" (UID: "fe7c1d18-2aaa-4338-91f1-9372ddfde274"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:14:27.284287 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284246    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe7c1d18-2aaa-4338-91f1-9372ddfde274" (UID: "fe7c1d18-2aaa-4338-91f1-9372ddfde274"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:14:27.284520 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284384    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe7c1d18-2aaa-4338-91f1-9372ddfde274" (UID: "fe7c1d18-2aaa-4338-91f1-9372ddfde274"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:14:27.284520 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284445    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe7c1d18-2aaa-4338-91f1-9372ddfde274" (UID: "fe7c1d18-2aaa-4338-91f1-9372ddfde274"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:14:27.284623 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284605    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:27.284664 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284629    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:27.284664 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284644    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:27.284664 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284656    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:27.284787 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.284668    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:27.286100 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.286073    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe7c1d18-2aaa-4338-91f1-9372ddfde274" (UID: "fe7c1d18-2aaa-4338-91f1-9372ddfde274"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:14:27.286212 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.286193    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe7c1d18-2aaa-4338-91f1-9372ddfde274" (UID: "fe7c1d18-2aaa-4338-91f1-9372ddfde274"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:14:27.286562 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.286538    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-kube-api-access-lmhj4" (OuterVolumeSpecName: "kube-api-access-lmhj4") pod "fe7c1d18-2aaa-4338-91f1-9372ddfde274" (UID: "fe7c1d18-2aaa-4338-91f1-9372ddfde274"). InnerVolumeSpecName "kube-api-access-lmhj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:14:27.286631 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.286561    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe7c1d18-2aaa-4338-91f1-9372ddfde274" (UID: "fe7c1d18-2aaa-4338-91f1-9372ddfde274"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:14:27.385864 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.385831    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:27.385864 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.385858    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:27.385864 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.385868    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmhj4\" (UniqueName: \"kubernetes.io/projected/fe7c1d18-2aaa-4338-91f1-9372ddfde274-kube-api-access-lmhj4\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:27.386092 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:27.385879    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe7c1d18-2aaa-4338-91f1-9372ddfde274-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:14:28.031961 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:28.031923    2576 generic.go:356] "Generic (PLEG): container finished" podID="fe7c1d18-2aaa-4338-91f1-9372ddfde274" containerID="7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b" exitCode=0
Jul 10 03:14:28.032169 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:28.031968    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"
Jul 10 03:14:28.032169 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:28.032008    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747" event={"ID":"fe7c1d18-2aaa-4338-91f1-9372ddfde274","Type":"ContainerDied","Data":"7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b"}
Jul 10 03:14:28.032169 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:28.032061    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747" event={"ID":"fe7c1d18-2aaa-4338-91f1-9372ddfde274","Type":"ContainerDied","Data":"670e39206d55ea4393f5984e6a36427333f60eefbcb1c81b196755a3df267fc0"}
Jul 10 03:14:28.032169 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:28.032085    2576 scope.go:117] "RemoveContainer" containerID="7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b"
Jul 10 03:14:28.041770 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:28.041727    2576 scope.go:117] "RemoveContainer" containerID="7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b"
Jul 10 03:14:28.042073 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:14:28.042046    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b\": container with ID starting with 7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b not found: ID does not exist" containerID="7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b"
Jul 10 03:14:28.042147 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:28.042076    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b"} err="failed to get container status \"7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b\": rpc error: code = NotFound desc = could not find container \"7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b\": container with ID starting with 7d34758fc8f559eff5707d8a01903b16bea5942eb419089cbfafe3166e6d508b not found: ID does not exist"
Jul 10 03:14:28.057144 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:28.057107    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"]
Jul 10 03:14:28.061393 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:28.061368    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-5374048d-openshift-default-855cb98967-hv747"]
Jul 10 03:14:28.799309 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:28.799274    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe7c1d18-2aaa-4338-91f1-9372ddfde274" path="/var/lib/kubelet/pods/fe7c1d18-2aaa-4338-91f1-9372ddfde274/volumes"
Jul 10 03:14:50.782438 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:50.782408    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:14:50.786251 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:14:50.786231    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:15:00.143519 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:00.143482    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4"]
Jul 10 03:15:00.144015 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:00.143895    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe7c1d18-2aaa-4338-91f1-9372ddfde274" containerName="istio-proxy"
Jul 10 03:15:00.144015 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:00.143907    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe7c1d18-2aaa-4338-91f1-9372ddfde274" containerName="istio-proxy"
Jul 10 03:15:00.144015 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:00.143982    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe7c1d18-2aaa-4338-91f1-9372ddfde274" containerName="istio-proxy"
Jul 10 03:15:00.144015 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:00.143992    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="68609a3d-24a1-41aa-a194-889734c45276" containerName="istio-proxy"
Jul 10 03:15:00.147025 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:00.147006    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4"
Jul 10 03:15:00.150357 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:00.150341    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nhxqz\""
Jul 10 03:15:00.157108 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:00.157078    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4"
Jul 10 03:15:00.158182 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:00.158162    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4"]
Jul 10 03:15:00.296217 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:00.296190    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4"]
Jul 10 03:15:00.298979 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:15:00.298953    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f3f9167_1e45_4ade_ab05_c9482bd221f3.slice/crio-5ba0c663d83a9144b053a2ae5016f7747d5b24f6d987fa38d6ecb89fa6825847 WatchSource:0}: Error finding container 5ba0c663d83a9144b053a2ae5016f7747d5b24f6d987fa38d6ecb89fa6825847: Status 404 returned error can't find the container with id 5ba0c663d83a9144b053a2ae5016f7747d5b24f6d987fa38d6ecb89fa6825847
Jul 10 03:15:01.206179 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:01.206142    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4" event={"ID":"7f3f9167-1e45-4ade-ab05-c9482bd221f3","Type":"ContainerStarted","Data":"fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10"}
Jul 10 03:15:01.206652 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:01.206194    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4" event={"ID":"7f3f9167-1e45-4ade-ab05-c9482bd221f3","Type":"ContainerStarted","Data":"5ba0c663d83a9144b053a2ae5016f7747d5b24f6d987fa38d6ecb89fa6825847"}
Jul 10 03:15:01.222739 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:15:01.222690    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4" podStartSLOduration=1.2226740600000001 podStartE2EDuration="1.22267406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:15:01.220895405 +0000 UTC m=+2111.030160715" watchObservedRunningTime="2026-07-10 03:15:01.22267406 +0000 UTC m=+2111.031939394"
Jul 10 03:16:00.011037 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:00.010953    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4"]
Jul 10 03:16:00.011520 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:00.011154    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4" podUID="7f3f9167-1e45-4ade-ab05-c9482bd221f3" containerName="cleanup" containerID="cri-o://fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10" gracePeriod=30
Jul 10 03:16:05.014839 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.014807    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"]
Jul 10 03:16:05.018443 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.018420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.021033 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.021012    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1924c46c-openshift-default-dockercfg-tfjhk\""
Jul 10 03:16:05.033995 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.033975    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"]
Jul 10 03:16:05.114184 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.114144    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3067377-ff64-4a8c-88ac-97bc855a4df9-istiod-ca-cert\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.114374 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.114212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-credential-socket\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.114374 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.114239    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-certs\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.114374 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.114309    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-data\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.114374 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.114336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-podinfo\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.114374 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.114360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c755g\" (UniqueName: \"kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-kube-api-access-c755g\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.114642 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.114412    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-token\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.114642 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.114471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-socket\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.114642 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.114573    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-envoy\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215280 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215248    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-token\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215446 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215299    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-socket\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215446 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-envoy\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215532 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3067377-ff64-4a8c-88ac-97bc855a4df9-istiod-ca-cert\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215532 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-credential-socket\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215634 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-certs\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215634 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215597    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-data\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215634 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215624    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-podinfo\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215835 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c755g\" (UniqueName: \"kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-kube-api-access-c755g\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215835 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215654    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-socket\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.215971 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.215949    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-certs\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.216037 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.216003    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-credential-socket\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.216097 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.216082    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-data\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.216320 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.216298    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3067377-ff64-4a8c-88ac-97bc855a4df9-istiod-ca-cert\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.217644 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.217611    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-envoy\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.218001 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.217983    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-podinfo\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.223237 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.223214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-token\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.223983 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.223956    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c755g\" (UniqueName: \"kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-kube-api-access-c755g\") pod \"e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") " pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.330797 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.330688    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:05.483430 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.483405    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"]
Jul 10 03:16:05.485094 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:16:05.485065    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3067377_ff64_4a8c_88ac_97bc855a4df9.slice/crio-7a5ff35dc26d03bc9b722497c87888315374862b8bc2fb632b67d8f078a7e39a WatchSource:0}: Error finding container 7a5ff35dc26d03bc9b722497c87888315374862b8bc2fb632b67d8f078a7e39a: Status 404 returned error can't find the container with id 7a5ff35dc26d03bc9b722497c87888315374862b8bc2fb632b67d8f078a7e39a
Jul 10 03:16:05.487344 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.487313    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:16:05.487463 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.487394    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:16:05.487463 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.487439    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:16:05.533921 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:05.533883    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" event={"ID":"b3067377-ff64-4a8c-88ac-97bc855a4df9","Type":"ContainerStarted","Data":"7a5ff35dc26d03bc9b722497c87888315374862b8bc2fb632b67d8f078a7e39a"}
Jul 10 03:16:06.542259 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:06.542218    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" event={"ID":"b3067377-ff64-4a8c-88ac-97bc855a4df9","Type":"ContainerStarted","Data":"8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652"}
Jul 10 03:16:06.567692 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:06.567539    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" podStartSLOduration=2.567519648 podStartE2EDuration="2.567519648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:16:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:16:06.567001215 +0000 UTC m=+2176.376266517" watchObservedRunningTime="2026-07-10 03:16:06.567519648 +0000 UTC m=+2176.376784951"
Jul 10 03:16:07.331057 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:07.331024    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:07.336255 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:07.336224    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:07.547075 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:07.547045    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:07.548180 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:07.548160    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:22.548901 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:22.548865    2576 patch_prober.go:28] interesting pod/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 10 03:16:22.549299 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:22.548933    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 10 03:16:22.628311 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:22.628279    2576 generic.go:356] "Generic (PLEG): container finished" podID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerID="8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652" exitCode=0
Jul 10 03:16:22.628474 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:22.628349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" event={"ID":"b3067377-ff64-4a8c-88ac-97bc855a4df9","Type":"ContainerDied","Data":"8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652"}
Jul 10 03:16:22.628794 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:22.628779    2576 scope.go:117] "RemoveContainer" containerID="8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652"
Jul 10 03:16:22.629826 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:22.629797    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:16:22.629934 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:22.629879    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:16:22.629934 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:22.629928    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:16:23.643020 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:23.642977    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" event={"ID":"b3067377-ff64-4a8c-88ac-97bc855a4df9","Type":"ContainerStarted","Data":"47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f"}
Jul 10 03:16:24.330931 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:24.330891    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:24.331309 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:24.331283    2576 patch_prober.go:28] interesting pod/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 10 03:16:24.331498 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:24.331470    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 10 03:16:25.331896 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:25.331849    2576 patch_prober.go:28] interesting pod/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 10 03:16:25.332393 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:25.331918    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 10 03:16:26.332228 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:26.331784    2576 patch_prober.go:28] interesting pod/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 10 03:16:26.332228 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:26.331846    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 10 03:16:27.331667 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:27.331626    2576 patch_prober.go:28] interesting pod/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 10 03:16:27.331994 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:27.331961    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 10 03:16:28.335480 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:28.335449    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:28.335848 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:28.335703    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:28.336420 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:28.336402    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:16:30.184644 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.184619    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4"
Jul 10 03:16:30.680647 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.680610    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f3f9167-1e45-4ade-ab05-c9482bd221f3" containerID="fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10" exitCode=137
Jul 10 03:16:30.680847 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.680659    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4"
Jul 10 03:16:30.680847 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.680699    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4" event={"ID":"7f3f9167-1e45-4ade-ab05-c9482bd221f3","Type":"ContainerDied","Data":"fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10"}
Jul 10 03:16:30.680847 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.680782    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4" event={"ID":"7f3f9167-1e45-4ade-ab05-c9482bd221f3","Type":"ContainerDied","Data":"5ba0c663d83a9144b053a2ae5016f7747d5b24f6d987fa38d6ecb89fa6825847"}
Jul 10 03:16:30.680847 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.680808    2576 scope.go:117] "RemoveContainer" containerID="fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10"
Jul 10 03:16:30.690277 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.690253    2576 scope.go:117] "RemoveContainer" containerID="fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10"
Jul 10 03:16:30.690531 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:16:30.690511    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10\": container with ID starting with fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10 not found: ID does not exist" containerID="fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10"
Jul 10 03:16:30.690600 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.690538    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10"} err="failed to get container status \"fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10\": rpc error: code = NotFound desc = could not find container \"fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10\": container with ID starting with fecac9687abfbe1d010318babef527f6f41e6e63ef50346f4755e2bccc82ac10 not found: ID does not exist"
Jul 10 03:16:30.702629 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.702604    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4"]
Jul 10 03:16:30.706725 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.706701    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727555-c6qb4"]
Jul 10 03:16:30.798832 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:16:30.798800    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f3f9167-1e45-4ade-ab05-c9482bd221f3" path="/var/lib/kubelet/pods/7f3f9167-1e45-4ade-ab05-c9482bd221f3/volumes"
Jul 10 03:17:33.248213 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:33.248131    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"]
Jul 10 03:17:33.248779 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:33.248377    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy" containerID="cri-o://47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f" gracePeriod=30
Jul 10 03:17:38.511946 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.511910    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:17:38.658826 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.658729    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-data\") pod \"b3067377-ff64-4a8c-88ac-97bc855a4df9\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") "
Jul 10 03:17:38.658826 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.658799    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-token\") pod \"b3067377-ff64-4a8c-88ac-97bc855a4df9\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") "
Jul 10 03:17:38.658826 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.658828    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c755g\" (UniqueName: \"kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-kube-api-access-c755g\") pod \"b3067377-ff64-4a8c-88ac-97bc855a4df9\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") "
Jul 10 03:17:38.659099 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.658851    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3067377-ff64-4a8c-88ac-97bc855a4df9-istiod-ca-cert\") pod \"b3067377-ff64-4a8c-88ac-97bc855a4df9\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") "
Jul 10 03:17:38.659099 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.658903    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-podinfo\") pod \"b3067377-ff64-4a8c-88ac-97bc855a4df9\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") "
Jul 10 03:17:38.659099 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.658931    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-certs\") pod \"b3067377-ff64-4a8c-88ac-97bc855a4df9\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") "
Jul 10 03:17:38.659099 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659054    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-credential-socket\") pod \"b3067377-ff64-4a8c-88ac-97bc855a4df9\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") "
Jul 10 03:17:38.659314 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659107    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-envoy\") pod \"b3067377-ff64-4a8c-88ac-97bc855a4df9\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") "
Jul 10 03:17:38.659314 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659147    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-socket\") pod \"b3067377-ff64-4a8c-88ac-97bc855a4df9\" (UID: \"b3067377-ff64-4a8c-88ac-97bc855a4df9\") "
Jul 10 03:17:38.659314 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659234    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b3067377-ff64-4a8c-88ac-97bc855a4df9" (UID: "b3067377-ff64-4a8c-88ac-97bc855a4df9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:17:38.659314 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659295    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3067377-ff64-4a8c-88ac-97bc855a4df9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b3067377-ff64-4a8c-88ac-97bc855a4df9" (UID: "b3067377-ff64-4a8c-88ac-97bc855a4df9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:17:38.659534 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659448    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b3067377-ff64-4a8c-88ac-97bc855a4df9" (UID: "b3067377-ff64-4a8c-88ac-97bc855a4df9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:17:38.659597 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659547    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3067377-ff64-4a8c-88ac-97bc855a4df9-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:17:38.659597 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659552    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-data" (OuterVolumeSpecName: "istio-data") pod "b3067377-ff64-4a8c-88ac-97bc855a4df9" (UID: "b3067377-ff64-4a8c-88ac-97bc855a4df9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:17:38.659597 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659567    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:17:38.659597 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659580    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:17:38.659866 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.659673    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b3067377-ff64-4a8c-88ac-97bc855a4df9" (UID: "b3067377-ff64-4a8c-88ac-97bc855a4df9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:17:38.661180 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.661149    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-kube-api-access-c755g" (OuterVolumeSpecName: "kube-api-access-c755g") pod "b3067377-ff64-4a8c-88ac-97bc855a4df9" (UID: "b3067377-ff64-4a8c-88ac-97bc855a4df9"). InnerVolumeSpecName "kube-api-access-c755g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:17:38.661275 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.661183    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b3067377-ff64-4a8c-88ac-97bc855a4df9" (UID: "b3067377-ff64-4a8c-88ac-97bc855a4df9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:17:38.661275 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.661185    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-token" (OuterVolumeSpecName: "istio-token") pod "b3067377-ff64-4a8c-88ac-97bc855a4df9" (UID: "b3067377-ff64-4a8c-88ac-97bc855a4df9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:17:38.661399 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.661380    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b3067377-ff64-4a8c-88ac-97bc855a4df9" (UID: "b3067377-ff64-4a8c-88ac-97bc855a4df9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:17:38.760685 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.760641    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:17:38.760685 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.760677    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:17:38.760685 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.760687    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c755g\" (UniqueName: \"kubernetes.io/projected/b3067377-ff64-4a8c-88ac-97bc855a4df9-kube-api-access-c755g\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:17:38.760685 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.760695    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:17:38.760972 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.760704    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:17:38.760972 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:38.760712    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3067377-ff64-4a8c-88ac-97bc855a4df9-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:17:39.035559 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.035521    2576 generic.go:356] "Generic (PLEG): container finished" podID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerID="47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f" exitCode=0
Jul 10 03:17:39.035734 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.035573    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"
Jul 10 03:17:39.035734 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.035604    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" event={"ID":"b3067377-ff64-4a8c-88ac-97bc855a4df9","Type":"ContainerDied","Data":"47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f"}
Jul 10 03:17:39.035734 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.035662    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb" event={"ID":"b3067377-ff64-4a8c-88ac-97bc855a4df9","Type":"ContainerDied","Data":"7a5ff35dc26d03bc9b722497c87888315374862b8bc2fb632b67d8f078a7e39a"}
Jul 10 03:17:39.035734 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.035685    2576 scope.go:117] "RemoveContainer" containerID="47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f"
Jul 10 03:17:39.051154 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.051133    2576 scope.go:117] "RemoveContainer" containerID="8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652"
Jul 10 03:17:39.059768 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.059728    2576 scope.go:117] "RemoveContainer" containerID="47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f"
Jul 10 03:17:39.059954 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.059938    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"]
Jul 10 03:17:39.060056 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:17:39.060035    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f\": container with ID starting with 47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f not found: ID does not exist" containerID="47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f"
Jul 10 03:17:39.060096 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.060065    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f"} err="failed to get container status \"47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f\": rpc error: code = NotFound desc = could not find container \"47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f\": container with ID starting with 47c286986a3b860ab7444d633865b424577f2ac598499c53910f1171776a6c6f not found: ID does not exist"
Jul 10 03:17:39.060096 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.060082    2576 scope.go:117] "RemoveContainer" containerID="8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652"
Jul 10 03:17:39.060313 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:17:39.060297    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652\": container with ID starting with 8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652 not found: ID does not exist" containerID="8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652"
Jul 10 03:17:39.060349 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.060318    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652"} err="failed to get container status \"8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652\": rpc error: code = NotFound desc = could not find container \"8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652\": container with ID starting with 8a80d0ed1e0331878b75ee158dffacec0a6dff9286b9d152bd41a50557bf2652 not found: ID does not exist"
Jul 10 03:17:39.063790 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:39.063767    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1924c46c-openshift-default-b4978dc99-lb7xb"]
Jul 10 03:17:40.799939 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:40.799904    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" path="/var/lib/kubelet/pods/b3067377-ff64-4a8c-88ac-97bc855a4df9/volumes"
Jul 10 03:17:53.449256 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.449216    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"]
Jul 10 03:17:53.449640 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.449627    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f3f9167-1e45-4ade-ab05-c9482bd221f3" containerName="cleanup"
Jul 10 03:17:53.449640 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.449640    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f3f9167-1e45-4ade-ab05-c9482bd221f3" containerName="cleanup"
Jul 10 03:17:53.449718 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.449655    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy"
Jul 10 03:17:53.449718 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.449661    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy"
Jul 10 03:17:53.449816 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.449735    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f3f9167-1e45-4ade-ab05-c9482bd221f3" containerName="cleanup"
Jul 10 03:17:53.449816 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.449770    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy"
Jul 10 03:17:53.449816 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.449778    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy"
Jul 10 03:17:53.452477 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.452460    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.455149 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.455125    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-57198657-openshift-default-dockercfg-rmq7b\""
Jul 10 03:17:53.462373 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.462350    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"]
Jul 10 03:17:53.496808 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.496776    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc65f427-37e2-4879-a14f-001a91d1756b-istiod-ca-cert\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.496960 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.496812    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-certs\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.496960 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.496852    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc65f427-37e2-4879-a14f-001a91d1756b-istio-podinfo\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.496960 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.496871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-envoy\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.496960 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.496909    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqstv\" (UniqueName: \"kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-kube-api-access-wqstv\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.497102 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.496970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-data\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.497102 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.497042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-credential-socket\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.497102 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.497080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-istio-token\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.497196 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.497105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-socket\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598040 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-socket\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598221 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598095    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc65f427-37e2-4879-a14f-001a91d1756b-istiod-ca-cert\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598221 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-certs\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598221 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc65f427-37e2-4879-a14f-001a91d1756b-istio-podinfo\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598221 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-envoy\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598409 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqstv\" (UniqueName: \"kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-kube-api-access-wqstv\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598409 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598275    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-data\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598409 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598340    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-credential-socket\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598409 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-istio-token\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598597 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598514    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-certs\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598597 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598585    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-socket\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598863 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598842    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-credential-socket\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.598863 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598846    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-data\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.599057 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.598923    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc65f427-37e2-4879-a14f-001a91d1756b-istiod-ca-cert\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.601122 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.601094    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-envoy\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.601550 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.601522    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc65f427-37e2-4879-a14f-001a91d1756b-istio-podinfo\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.606052 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.606022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-istio-token\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.606465 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.606445    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqstv\" (UniqueName: \"kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-kube-api-access-wqstv\") pod \"e2e-mt-57198657-openshift-default-6d5599b699-8j5d8\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") " pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.764247 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.764205    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:53.902172 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.902146    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"]
Jul 10 03:17:53.905146 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:17:53.905123    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc65f427_37e2_4879_a14f_001a91d1756b.slice/crio-5093403bb778b47bad45bde4b164e4c0b29a377121b7ae3f296b7048aaa796de WatchSource:0}: Error finding container 5093403bb778b47bad45bde4b164e4c0b29a377121b7ae3f296b7048aaa796de: Status 404 returned error can't find the container with id 5093403bb778b47bad45bde4b164e4c0b29a377121b7ae3f296b7048aaa796de
Jul 10 03:17:53.907997 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.907959    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:17:53.908085 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.908029    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:17:53.908085 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:53.908059    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:17:54.111798 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:54.111688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" event={"ID":"dc65f427-37e2-4879-a14f-001a91d1756b","Type":"ContainerStarted","Data":"3a3c30de669230880ca1587d6bc13d0e6bd371901ba83095d78026e260c78d7f"}
Jul 10 03:17:54.111798 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:54.111759    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" event={"ID":"dc65f427-37e2-4879-a14f-001a91d1756b","Type":"ContainerStarted","Data":"5093403bb778b47bad45bde4b164e4c0b29a377121b7ae3f296b7048aaa796de"}
Jul 10 03:17:54.131233 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:54.131187    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" podStartSLOduration=1.131170658 podStartE2EDuration="1.131170658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:17:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:17:54.128535114 +0000 UTC m=+2283.937800419" watchObservedRunningTime="2026-07-10 03:17:54.131170658 +0000 UTC m=+2283.940435959"
Jul 10 03:17:54.764798 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:54.764760    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:54.769819 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:54.769791    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:55.117466 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:55.117379    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:17:55.118302 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:17:55.118282    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:18:11.157975 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:11.157932    2576 patch_prober.go:28] interesting pod/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 03:18:11.158518 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:11.158018    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 03:18:12.202700 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:12.202662    2576 generic.go:356] "Generic (PLEG): container finished" podID="dc65f427-37e2-4879-a14f-001a91d1756b" containerID="3a3c30de669230880ca1587d6bc13d0e6bd371901ba83095d78026e260c78d7f" exitCode=0
Jul 10 03:18:12.203237 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:12.202773    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" event={"ID":"dc65f427-37e2-4879-a14f-001a91d1756b","Type":"ContainerDied","Data":"3a3c30de669230880ca1587d6bc13d0e6bd371901ba83095d78026e260c78d7f"}
Jul 10 03:18:12.203309 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:12.203292    2576 scope.go:117] "RemoveContainer" containerID="3a3c30de669230880ca1587d6bc13d0e6bd371901ba83095d78026e260c78d7f"
Jul 10 03:18:12.204326 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:12.204297    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:18:12.204386 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:12.204365    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:18:12.204422 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:12.204396    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:18:13.212326 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:13.212291    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" event={"ID":"dc65f427-37e2-4879-a14f-001a91d1756b","Type":"ContainerStarted","Data":"829a7b1c885301cc919a9a4fb698bef139978d94e591c8209d0cbe4dafb56c01"}
Jul 10 03:18:13.765343 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:13.765307    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:18:13.765595 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:13.765572    2576 patch_prober.go:28] interesting pod/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 10 03:18:13.765650 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:13.765631    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 10 03:18:14.765359 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:14.765322    2576 patch_prober.go:28] interesting pod/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 10 03:18:14.765785 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:14.765384    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 10 03:18:15.765473 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:15.765432    2576 patch_prober.go:28] interesting pod/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 10 03:18:15.765880 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:15.765493    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 10 03:18:16.765198 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:16.765158    2576 patch_prober.go:28] interesting pod/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 10 03:18:16.765387 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:16.765219    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 10 03:18:17.768936 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:17.768901    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:18:17.769418 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:17.769270    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:18:17.770128 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:18:17.770107    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:19:50.819385 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:19:50.819340    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:19:50.823966 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:19:50.823947    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:24:45.479778 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:45.479679    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"]
Jul 10 03:24:45.480230 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:45.479968    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy" containerID="cri-o://829a7b1c885301cc919a9a4fb698bef139978d94e591c8209d0cbe4dafb56c01" gracePeriod=30
Jul 10 03:24:47.771862 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:47.771823    2576 patch_prober.go:28] interesting pod/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 03:24:47.772230 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:47.771891    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 03:24:50.739610 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.739584    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:24:50.802621 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802544    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc65f427-37e2-4879-a14f-001a91d1756b-istiod-ca-cert\") pod \"dc65f427-37e2-4879-a14f-001a91d1756b\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") "
Jul 10 03:24:50.802621 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802584    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-certs\") pod \"dc65f427-37e2-4879-a14f-001a91d1756b\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") "
Jul 10 03:24:50.802621 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802603    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqstv\" (UniqueName: \"kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-kube-api-access-wqstv\") pod \"dc65f427-37e2-4879-a14f-001a91d1756b\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") "
Jul 10 03:24:50.802910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802629    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-socket\") pod \"dc65f427-37e2-4879-a14f-001a91d1756b\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") "
Jul 10 03:24:50.802910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802646    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-istio-token\") pod \"dc65f427-37e2-4879-a14f-001a91d1756b\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") "
Jul 10 03:24:50.802910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802677    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-data\") pod \"dc65f427-37e2-4879-a14f-001a91d1756b\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") "
Jul 10 03:24:50.802910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802710    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-envoy\") pod \"dc65f427-37e2-4879-a14f-001a91d1756b\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") "
Jul 10 03:24:50.802910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-credential-socket\") pod \"dc65f427-37e2-4879-a14f-001a91d1756b\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") "
Jul 10 03:24:50.802910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802810    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc65f427-37e2-4879-a14f-001a91d1756b-istio-podinfo\") pod \"dc65f427-37e2-4879-a14f-001a91d1756b\" (UID: \"dc65f427-37e2-4879-a14f-001a91d1756b\") "
Jul 10 03:24:50.803224 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802917    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc65f427-37e2-4879-a14f-001a91d1756b" (UID: "dc65f427-37e2-4879-a14f-001a91d1756b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:24:50.803224 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.802978    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc65f427-37e2-4879-a14f-001a91d1756b" (UID: "dc65f427-37e2-4879-a14f-001a91d1756b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:24:50.803224 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.803014    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc65f427-37e2-4879-a14f-001a91d1756b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc65f427-37e2-4879-a14f-001a91d1756b" (UID: "dc65f427-37e2-4879-a14f-001a91d1756b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:24:50.803224 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.803142    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc65f427-37e2-4879-a14f-001a91d1756b-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:24:50.803224 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.803159    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:24:50.803224 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.803172    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:24:50.803224 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.803212    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc65f427-37e2-4879-a14f-001a91d1756b" (UID: "dc65f427-37e2-4879-a14f-001a91d1756b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:24:50.803793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.803771    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc65f427-37e2-4879-a14f-001a91d1756b" (UID: "dc65f427-37e2-4879-a14f-001a91d1756b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:24:50.805387 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.805355    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-kube-api-access-wqstv" (OuterVolumeSpecName: "kube-api-access-wqstv") pod "dc65f427-37e2-4879-a14f-001a91d1756b" (UID: "dc65f427-37e2-4879-a14f-001a91d1756b"). InnerVolumeSpecName "kube-api-access-wqstv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:24:50.805489 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.805401    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc65f427-37e2-4879-a14f-001a91d1756b" (UID: "dc65f427-37e2-4879-a14f-001a91d1756b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:24:50.805489 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.805418    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc65f427-37e2-4879-a14f-001a91d1756b" (UID: "dc65f427-37e2-4879-a14f-001a91d1756b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:24:50.805489 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.805433    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc65f427-37e2-4879-a14f-001a91d1756b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc65f427-37e2-4879-a14f-001a91d1756b" (UID: "dc65f427-37e2-4879-a14f-001a91d1756b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:24:50.855559 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.855531    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:24:50.860256 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.860233    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:24:50.904493 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.904456    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:24:50.904493 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.904485    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc65f427-37e2-4879-a14f-001a91d1756b-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:24:50.904493 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.904498    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqstv\" (UniqueName: \"kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-kube-api-access-wqstv\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:24:50.904720 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.904516    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc65f427-37e2-4879-a14f-001a91d1756b-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:24:50.904720 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.904526    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:24:50.904720 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.904533    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc65f427-37e2-4879-a14f-001a91d1756b-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:24:50.933909 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.933882    2576 scope.go:117] "RemoveContainer" containerID="3a3c30de669230880ca1587d6bc13d0e6bd371901ba83095d78026e260c78d7f"
Jul 10 03:24:50.942972 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:50.942948    2576 scope.go:117] "RemoveContainer" containerID="829a7b1c885301cc919a9a4fb698bef139978d94e591c8209d0cbe4dafb56c01"
Jul 10 03:24:51.264266 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:51.264231    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"
Jul 10 03:24:51.264443 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:51.264223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" event={"ID":"dc65f427-37e2-4879-a14f-001a91d1756b","Type":"ContainerDied","Data":"829a7b1c885301cc919a9a4fb698bef139978d94e591c8209d0cbe4dafb56c01"}
Jul 10 03:24:51.264443 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:51.264374    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8" event={"ID":"dc65f427-37e2-4879-a14f-001a91d1756b","Type":"ContainerDied","Data":"5093403bb778b47bad45bde4b164e4c0b29a377121b7ae3f296b7048aaa796de"}
Jul 10 03:24:51.299188 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:51.299155    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"]
Jul 10 03:24:51.303177 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:51.303155    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-57198657-openshift-default-6d5599b699-8j5d8"]
Jul 10 03:24:52.799389 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:24:52.799347    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" path="/var/lib/kubelet/pods/dc65f427-37e2-4879-a14f-001a91d1756b/volumes"
Jul 10 03:25:43.289321 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.289283    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"]
Jul 10 03:25:43.289771 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.289737    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy"
Jul 10 03:25:43.289771 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.289772    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3067377-ff64-4a8c-88ac-97bc855a4df9" containerName="istio-proxy"
Jul 10 03:25:43.289865 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.289799    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy"
Jul 10 03:25:43.289865 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.289805    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy"
Jul 10 03:25:43.289865 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.289817    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy"
Jul 10 03:25:43.289865 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.289823    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy"
Jul 10 03:25:43.289988 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.289880    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy"
Jul 10 03:25:43.289988 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.289889    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc65f427-37e2-4879-a14f-001a91d1756b" containerName="istio-proxy"
Jul 10 03:25:43.292954 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.292935    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.295400 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.295381    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e7f792-openshift-default-dockercfg-hqt7j\""
Jul 10 03:25:43.304114 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.304088    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"]
Jul 10 03:25:43.306499 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.306475    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-credential-socket\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.306591 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.306514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-certs\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.306591 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.306532    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-podinfo\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.306591 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.306551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-envoy\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.306713 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.306598    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd84587-f900-4a26-9f3e-f0462f953f40-istiod-ca-cert\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.306713 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.306614    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-data\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.306713 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.306642    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-socket\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.306713 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.306659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgq5j\" (UniqueName: \"kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-kube-api-access-kgq5j\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.306713 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.306696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-token\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.407316 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407275    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-envoy\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.407517 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd84587-f900-4a26-9f3e-f0462f953f40-istiod-ca-cert\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.407517 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-data\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.407517 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407431    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-socket\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.407517 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgq5j\" (UniqueName: \"kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-kube-api-access-kgq5j\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.407742 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407547    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-token\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.407742 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-credential-socket\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.407742 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-certs\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.407742 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-podinfo\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.408051 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-socket\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.408051 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.407931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-data\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.408177 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.408151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-credential-socket\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.408262 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.408241    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-certs\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.408303 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.408258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd84587-f900-4a26-9f3e-f0462f953f40-istiod-ca-cert\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.409730 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.409710    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-envoy\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.410164 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.410147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-podinfo\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.415478 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.415457    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgq5j\" (UniqueName: \"kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-kube-api-access-kgq5j\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.415559 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.415516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-token\") pod \"e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") " pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.604270 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.604184    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:43.749289 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.749263    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"]
Jul 10 03:25:43.751875 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:25:43.751845    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd84587_f900_4a26_9f3e_f0462f953f40.slice/crio-7055f318c595c0a13bb1788d77cccaf61c99800c431fd5c319341c346c45f59a WatchSource:0}: Error finding container 7055f318c595c0a13bb1788d77cccaf61c99800c431fd5c319341c346c45f59a: Status 404 returned error can't find the container with id 7055f318c595c0a13bb1788d77cccaf61c99800c431fd5c319341c346c45f59a
Jul 10 03:25:43.754115 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.754081    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:25:43.754229 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.754145    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:25:43.754229 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:43.754177    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:25:44.533022 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:44.532975    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" event={"ID":"3bd84587-f900-4a26-9f3e-f0462f953f40","Type":"ContainerStarted","Data":"e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12"}
Jul 10 03:25:44.533408 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:44.533048    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" event={"ID":"3bd84587-f900-4a26-9f3e-f0462f953f40","Type":"ContainerStarted","Data":"7055f318c595c0a13bb1788d77cccaf61c99800c431fd5c319341c346c45f59a"}
Jul 10 03:25:44.557678 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:44.557630    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" podStartSLOduration=1.557613082 podStartE2EDuration="1.557613082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:25:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:25:44.554975089 +0000 UTC m=+2754.364240394" watchObservedRunningTime="2026-07-10 03:25:44.557613082 +0000 UTC m=+2754.366878443"
Jul 10 03:25:44.604944 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:44.604905    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:44.610117 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:44.610092    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:45.538035 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:45.538003    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:45.538948 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:45.538927    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:25:51.755373 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.755334    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"]
Jul 10 03:25:51.760594 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.760572    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.763553 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.763533    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-48f01f-openshift-default-dockercfg-2x57c\""
Jul 10 03:25:51.770339 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.770317    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"]
Jul 10 03:25:51.794417 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.790987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-credential-socket\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.794417 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.791386    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-certs\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.794417 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.791426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-data\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.794417 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.791563    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-envoy\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.794417 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.791601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkxp9\" (UniqueName: \"kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-kube-api-access-jkxp9\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.794417 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.791648    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-podinfo\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.794417 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.791700    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istiod-ca-cert\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.794417 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.791762    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-socket\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.794417 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.791810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-token\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.892760 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.892721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-envoy\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.892962 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.892789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkxp9\" (UniqueName: \"kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-kube-api-access-jkxp9\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.892962 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.892813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-podinfo\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.892962 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.892843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istiod-ca-cert\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.892962 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.892879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-socket\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.893204 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.893101    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-token\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.893204 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.893178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-credential-socket\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.893301 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.893239    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-certs\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.893301 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.893257    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-data\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.893301 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.893276    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-socket\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.893548 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.893525    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-data\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.893837 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.893811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-credential-socket\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.894026 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.894008    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-certs\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.894487 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.894466    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istiod-ca-cert\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.895134 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.895120    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-envoy\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.895767 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.895736    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-podinfo\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.900491 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.900469    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-token\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:51.901000 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:51.900984    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkxp9\" (UniqueName: \"kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-kube-api-access-jkxp9\") pod \"e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") " pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:52.071471 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:52.071373    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:52.209113 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:52.208988    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"]
Jul 10 03:25:52.211513 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:25:52.211481    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8498a3bb_ed2f_487a_8648_1eba8d08fd24.slice/crio-2774ad9fa20cadce1bf06e5b157bfd826f743c8ce8debedad4c46b0613355ad7 WatchSource:0}: Error finding container 2774ad9fa20cadce1bf06e5b157bfd826f743c8ce8debedad4c46b0613355ad7: Status 404 returned error can't find the container with id 2774ad9fa20cadce1bf06e5b157bfd826f743c8ce8debedad4c46b0613355ad7
Jul 10 03:25:52.213947 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:52.213914    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:25:52.214054 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:52.213995    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:25:52.214054 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:52.214039    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:25:52.577938 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:52.577900    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" event={"ID":"8498a3bb-ed2f-487a-8648-1eba8d08fd24","Type":"ContainerStarted","Data":"11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a"}
Jul 10 03:25:52.578160 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:52.577961    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" event={"ID":"8498a3bb-ed2f-487a-8648-1eba8d08fd24","Type":"ContainerStarted","Data":"2774ad9fa20cadce1bf06e5b157bfd826f743c8ce8debedad4c46b0613355ad7"}
Jul 10 03:25:52.597674 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:52.597617    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" podStartSLOduration=1.597599202 podStartE2EDuration="1.597599202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:25:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:25:52.597026841 +0000 UTC m=+2762.406292150" watchObservedRunningTime="2026-07-10 03:25:52.597599202 +0000 UTC m=+2762.406864504"
Jul 10 03:25:53.072347 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:53.072309    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:53.077132 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:53.077107    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:53.583265 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:53.583235    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:25:53.584253 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:25:53.584231    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:26:01.580937 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:01.580892    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 03:26:01.581377 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:01.580985    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 03:26:03.641878 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:03.641842    2576 generic.go:356] "Generic (PLEG): container finished" podID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerID="e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12" exitCode=0
Jul 10 03:26:03.642392 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:03.641917    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" event={"ID":"3bd84587-f900-4a26-9f3e-f0462f953f40","Type":"ContainerDied","Data":"e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12"}
Jul 10 03:26:03.642461 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:03.642421    2576 scope.go:117] "RemoveContainer" containerID="e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12"
Jul 10 03:26:03.643767 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:03.643712    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:26:03.643881 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:03.643834    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:26:03.643957 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:03.643882    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:26:04.649650 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:04.649612    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" event={"ID":"3bd84587-f900-4a26-9f3e-f0462f953f40","Type":"ContainerStarted","Data":"37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492"}
Jul 10 03:26:05.605333 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:05.605296    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:26:05.605513 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:05.605427    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 10 03:26:05.605513 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:05.605478    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 10 03:26:06.604801 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:06.604739    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 10 03:26:06.605189 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:06.604822    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 10 03:26:07.604957 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:07.604922    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 10 03:26:07.605350 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:07.604977    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 10 03:26:08.605288 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:08.605252    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 10 03:26:08.605772 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:08.605316    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 10 03:26:09.608569 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:09.608533    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:26:09.609069 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:09.608949    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:26:09.609661 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:09.609642    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:26:09.624870 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:09.624846    2576 patch_prober.go:28] interesting pod/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 03:26:09.624980 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:09.624888    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 03:26:10.690991 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:10.690957    2576 generic.go:356] "Generic (PLEG): container finished" podID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerID="11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a" exitCode=0
Jul 10 03:26:10.691423 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:10.691026    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" event={"ID":"8498a3bb-ed2f-487a-8648-1eba8d08fd24","Type":"ContainerDied","Data":"11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a"}
Jul 10 03:26:10.691582 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:10.691565    2576 scope.go:117] "RemoveContainer" containerID="11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a"
Jul 10 03:26:10.692560 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:10.692530    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:26:10.692672 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:10.692611    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:26:10.692672 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:10.692662    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:26:11.697404 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:11.697369    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" event={"ID":"8498a3bb-ed2f-487a-8648-1eba8d08fd24","Type":"ContainerStarted","Data":"215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34"}
Jul 10 03:26:12.072378 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:12.072340    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:26:12.072591 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:12.072568    2576 patch_prober.go:28] interesting pod/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 03:26:12.072651 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:12.072632    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 03:26:13.072773 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:13.072711    2576 patch_prober.go:28] interesting pod/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 03:26:13.073183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:13.072795    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 03:26:14.072030 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:14.071994    2576 patch_prober.go:28] interesting pod/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 03:26:14.072225 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:14.072051    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 03:26:15.072828 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:15.072791    2576 patch_prober.go:28] interesting pod/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 03:26:15.073209 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:15.072864    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 03:26:16.075364 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:16.075329    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:26:16.075739 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:16.075719    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:26:16.076474 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:26:16.076454    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:27:13.021617 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.021584    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"]
Jul 10 03:27:13.024334 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.024316    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.026971 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.026948    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-reuse-0566e7-openshift-default-dockercfg-ntfm8\""
Jul 10 03:27:13.036334 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.036309    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"]
Jul 10 03:27:13.098187 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.098149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-envoy\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.098187 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.098188    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-socket\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.098465 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.098207    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-token\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.098465 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.098263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-certs\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.098465 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.098317    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-podinfo\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.098465 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.098430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-credential-socket\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.098465 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.098463    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7qwt\" (UniqueName: \"kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-kube-api-access-p7qwt\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.098651 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.098588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c844cc-1dd4-4896-be3d-cc2560789b75-istiod-ca-cert\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.098651 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.098639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-data\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.199559 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.199524    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-socket\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.199559 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.199562    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-token\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.199868 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.199585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-certs\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.199868 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.199609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-podinfo\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.199868 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.199682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-credential-socket\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.199868 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.199712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7qwt\" (UniqueName: \"kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-kube-api-access-p7qwt\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.199868 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.199818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c844cc-1dd4-4896-be3d-cc2560789b75-istiod-ca-cert\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.199868 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.199867    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-data\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.200170 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.200000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-envoy\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.200170 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.200055    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-socket\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.200392 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.200342    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-data\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.200541 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.200408    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-credential-socket\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.200606 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.200584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c844cc-1dd4-4896-be3d-cc2560789b75-istiod-ca-cert\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.200702 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.200684    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-certs\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.202089 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.202066    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-envoy\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.202315 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.202296    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-podinfo\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.247817 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.247781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-token\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.247965 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.247885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7qwt\" (UniqueName: \"kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-kube-api-access-p7qwt\") pod \"e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") " pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.335987 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.335892    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:13.479089 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.479063    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"]
Jul 10 03:27:13.480896 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:27:13.480871    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c844cc_1dd4_4896_be3d_cc2560789b75.slice/crio-474c7154d14e88be06ed6e5cc5c93e4ae667088c798248b2fbdd89a40ecd792b WatchSource:0}: Error finding container 474c7154d14e88be06ed6e5cc5c93e4ae667088c798248b2fbdd89a40ecd792b: Status 404 returned error can't find the container with id 474c7154d14e88be06ed6e5cc5c93e4ae667088c798248b2fbdd89a40ecd792b
Jul 10 03:27:13.483055 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.483025    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:27:13.483124 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.483087    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:27:13.483169 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:13.483122    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:27:14.011021 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:14.010986    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" event={"ID":"86c844cc-1dd4-4896-be3d-cc2560789b75","Type":"ContainerStarted","Data":"daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b"}
Jul 10 03:27:14.011211 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:14.011039    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" event={"ID":"86c844cc-1dd4-4896-be3d-cc2560789b75","Type":"ContainerStarted","Data":"474c7154d14e88be06ed6e5cc5c93e4ae667088c798248b2fbdd89a40ecd792b"}
Jul 10 03:27:14.032803 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:14.032728    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" podStartSLOduration=1.032711918 podStartE2EDuration="1.032711918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:27:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:27:14.031864759 +0000 UTC m=+2843.841130060" watchObservedRunningTime="2026-07-10 03:27:14.032711918 +0000 UTC m=+2843.841977219"
Jul 10 03:27:14.336601 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:14.336487    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:14.341586 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:14.341557    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:15.016555 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:15.016522    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:15.017529 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:15.017504    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:36.128433 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:36.128395    2576 generic.go:356] "Generic (PLEG): container finished" podID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerID="daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b" exitCode=0
Jul 10 03:27:36.128867 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:36.128452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" event={"ID":"86c844cc-1dd4-4896-be3d-cc2560789b75","Type":"ContainerDied","Data":"daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b"}
Jul 10 03:27:36.128937 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:36.128922    2576 scope.go:117] "RemoveContainer" containerID="daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b"
Jul 10 03:27:36.129988 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:36.129958    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:27:36.130061 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:36.130031    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:27:36.130103 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:36.130066    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:27:37.139397 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:37.139353    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" event={"ID":"86c844cc-1dd4-4896-be3d-cc2560789b75","Type":"ContainerStarted","Data":"38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c"}
Jul 10 03:27:37.336904 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:37.336857    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:37.337216 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:37.336929    2576 patch_prober.go:28] interesting pod/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 10 03:27:37.337437 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:37.337396    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 10 03:27:38.336883 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:38.336397    2576 patch_prober.go:28] interesting pod/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 10 03:27:38.336883 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:38.336461    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 10 03:27:39.337253 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:39.337206    2576 patch_prober.go:28] interesting pod/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 10 03:27:39.337773 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:39.337275    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 10 03:27:40.337214 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:40.337173    2576 patch_prober.go:28] interesting pod/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 10 03:27:40.337510 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:40.337480    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 10 03:27:41.340673 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:41.340637    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:41.341047 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:41.340963    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:27:41.341800 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:27:41.341783    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:29:29.415535 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:29.415499    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"]
Jul 10 03:29:29.416005 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:29.415727    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy" containerID="cri-o://38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c" gracePeriod=30
Jul 10 03:29:34.659155 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.659123    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:29:34.722646 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.722613    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-credential-socket\") pod \"86c844cc-1dd4-4896-be3d-cc2560789b75\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") "
Jul 10 03:29:34.722646 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.722651    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-data\") pod \"86c844cc-1dd4-4896-be3d-cc2560789b75\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") "
Jul 10 03:29:34.722926 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.722669    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c844cc-1dd4-4896-be3d-cc2560789b75-istiod-ca-cert\") pod \"86c844cc-1dd4-4896-be3d-cc2560789b75\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") "
Jul 10 03:29:34.722926 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.722790    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-envoy\") pod \"86c844cc-1dd4-4896-be3d-cc2560789b75\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") "
Jul 10 03:29:34.722926 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.722820    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-certs\") pod \"86c844cc-1dd4-4896-be3d-cc2560789b75\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") "
Jul 10 03:29:34.722926 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.722852    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-token\") pod \"86c844cc-1dd4-4896-be3d-cc2560789b75\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") "
Jul 10 03:29:34.722926 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.722879    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-socket\") pod \"86c844cc-1dd4-4896-be3d-cc2560789b75\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") "
Jul 10 03:29:34.722926 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.722908    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-podinfo\") pod \"86c844cc-1dd4-4896-be3d-cc2560789b75\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") "
Jul 10 03:29:34.723235 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.722947    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7qwt\" (UniqueName: \"kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-kube-api-access-p7qwt\") pod \"86c844cc-1dd4-4896-be3d-cc2560789b75\" (UID: \"86c844cc-1dd4-4896-be3d-cc2560789b75\") "
Jul 10 03:29:34.723235 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.723024    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "86c844cc-1dd4-4896-be3d-cc2560789b75" (UID: "86c844cc-1dd4-4896-be3d-cc2560789b75"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:29:34.723235 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.723134    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "86c844cc-1dd4-4896-be3d-cc2560789b75" (UID: "86c844cc-1dd4-4896-be3d-cc2560789b75"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:29:34.723380 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.723170    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86c844cc-1dd4-4896-be3d-cc2560789b75-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "86c844cc-1dd4-4896-be3d-cc2560789b75" (UID: "86c844cc-1dd4-4896-be3d-cc2560789b75"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:29:34.723380 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.723247    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "86c844cc-1dd4-4896-be3d-cc2560789b75" (UID: "86c844cc-1dd4-4896-be3d-cc2560789b75"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:29:34.723495 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.723417    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:29:34.723495 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.723436    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:29:34.723495 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.723449    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:29:34.723495 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.723465    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c844cc-1dd4-4896-be3d-cc2560789b75-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:29:34.723636 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.723497    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-data" (OuterVolumeSpecName: "istio-data") pod "86c844cc-1dd4-4896-be3d-cc2560789b75" (UID: "86c844cc-1dd4-4896-be3d-cc2560789b75"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:29:34.725054 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.725031    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "86c844cc-1dd4-4896-be3d-cc2560789b75" (UID: "86c844cc-1dd4-4896-be3d-cc2560789b75"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:29:34.725166 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.725137    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "86c844cc-1dd4-4896-be3d-cc2560789b75" (UID: "86c844cc-1dd4-4896-be3d-cc2560789b75"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:29:34.725556 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.725535    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-token" (OuterVolumeSpecName: "istio-token") pod "86c844cc-1dd4-4896-be3d-cc2560789b75" (UID: "86c844cc-1dd4-4896-be3d-cc2560789b75"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:29:34.725655 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.725559    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-kube-api-access-p7qwt" (OuterVolumeSpecName: "kube-api-access-p7qwt") pod "86c844cc-1dd4-4896-be3d-cc2560789b75" (UID: "86c844cc-1dd4-4896-be3d-cc2560789b75"). InnerVolumeSpecName "kube-api-access-p7qwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:29:34.751450 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.751420    2576 generic.go:356] "Generic (PLEG): container finished" podID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerID="38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c" exitCode=0
Jul 10 03:29:34.751610 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.751472    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"
Jul 10 03:29:34.751610 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.751516    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" event={"ID":"86c844cc-1dd4-4896-be3d-cc2560789b75","Type":"ContainerDied","Data":"38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c"}
Jul 10 03:29:34.751727 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.751610    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx" event={"ID":"86c844cc-1dd4-4896-be3d-cc2560789b75","Type":"ContainerDied","Data":"474c7154d14e88be06ed6e5cc5c93e4ae667088c798248b2fbdd89a40ecd792b"}
Jul 10 03:29:34.751727 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.751643    2576 scope.go:117] "RemoveContainer" containerID="38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c"
Jul 10 03:29:34.761321 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.761307    2576 scope.go:117] "RemoveContainer" containerID="daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b"
Jul 10 03:29:34.769488 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.769474    2576 scope.go:117] "RemoveContainer" containerID="38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c"
Jul 10 03:29:34.769733 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:29:34.769711    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c\": container with ID starting with 38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c not found: ID does not exist" containerID="38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c"
Jul 10 03:29:34.769828 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.769768    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c"} err="failed to get container status \"38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c\": rpc error: code = NotFound desc = could not find container \"38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c\": container with ID starting with 38df1de5866e31f0d4c546e5f08e9692799c3b615ab78b92db953b47a923485c not found: ID does not exist"
Jul 10 03:29:34.769828 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.769794    2576 scope.go:117] "RemoveContainer" containerID="daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b"
Jul 10 03:29:34.770117 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:29:34.770074    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b\": container with ID starting with daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b not found: ID does not exist" containerID="daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b"
Jul 10 03:29:34.770233 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.770124    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b"} err="failed to get container status \"daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b\": rpc error: code = NotFound desc = could not find container \"daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b\": container with ID starting with daa0ae43c2ee6ff8a41935920c2eadc87861c848b4f29ecf3e2636147b78ad6b not found: ID does not exist"
Jul 10 03:29:34.772861 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.772839    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"]
Jul 10 03:29:34.778216 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.778195    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-reuse-0566e7-openshift-default-64cd5577d9-bsbgx"]
Jul 10 03:29:34.799411 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.799381    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" path="/var/lib/kubelet/pods/86c844cc-1dd4-4896-be3d-cc2560789b75/volumes"
Jul 10 03:29:34.824048 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.824025    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:29:34.824048 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.824046    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:29:34.824190 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.824054    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:29:34.824190 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.824063    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7qwt\" (UniqueName: \"kubernetes.io/projected/86c844cc-1dd4-4896-be3d-cc2560789b75-kube-api-access-p7qwt\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:29:34.824190 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:34.824076    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c844cc-1dd4-4896-be3d-cc2560789b75-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:29:50.893481 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:50.893456    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:29:50.899426 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:29:50.899403    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:30:00.150206 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.150165    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t"]
Jul 10 03:30:00.150685 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.150603    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy"
Jul 10 03:30:00.150685 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.150615    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy"
Jul 10 03:30:00.150685 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.150636    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy"
Jul 10 03:30:00.150685 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.150641    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy"
Jul 10 03:30:00.150861 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.150720    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy"
Jul 10 03:30:00.150861 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.150729    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="86c844cc-1dd4-4896-be3d-cc2560789b75" containerName="istio-proxy"
Jul 10 03:30:00.154423 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.154400    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr"]
Jul 10 03:30:00.154567 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.154523    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t"
Jul 10 03:30:00.156759 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.156731    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr"
Jul 10 03:30:00.157053 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.157039    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nhxqz\""
Jul 10 03:30:00.161182 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.161157    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t"]
Jul 10 03:30:00.164957 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.164922    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr"]
Jul 10 03:30:00.168024 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.168005    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t"
Jul 10 03:30:00.175506 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.175490    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr"
Jul 10 03:30:00.315824 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.315792    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t"]
Jul 10 03:30:00.318253 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:30:00.318224    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9667e21b_36ff_4920_a149_a3ed9d30d5a3.slice/crio-ce0dfd3ff37bdf0c95d2ca8ad90b2336eeb63bb5f4b54905554092586a46a455 WatchSource:0}: Error finding container ce0dfd3ff37bdf0c95d2ca8ad90b2336eeb63bb5f4b54905554092586a46a455: Status 404 returned error can't find the container with id ce0dfd3ff37bdf0c95d2ca8ad90b2336eeb63bb5f4b54905554092586a46a455
Jul 10 03:30:00.336343 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.336315    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr"]
Jul 10 03:30:00.337120 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:30:00.337096    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedebe537_c2f9_4728_ab48_98c80d319e25.slice/crio-329b1097614741654435eb525ce76e2b1fcaaba4edc6360eb43c04444a865625 WatchSource:0}: Error finding container 329b1097614741654435eb525ce76e2b1fcaaba4edc6360eb43c04444a865625: Status 404 returned error can't find the container with id 329b1097614741654435eb525ce76e2b1fcaaba4edc6360eb43c04444a865625
Jul 10 03:30:00.894673 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.894582    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t" event={"ID":"9667e21b-36ff-4920-a149-a3ed9d30d5a3","Type":"ContainerStarted","Data":"c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf"}
Jul 10 03:30:00.894673 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.894642    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t" event={"ID":"9667e21b-36ff-4920-a149-a3ed9d30d5a3","Type":"ContainerStarted","Data":"ce0dfd3ff37bdf0c95d2ca8ad90b2336eeb63bb5f4b54905554092586a46a455"}
Jul 10 03:30:00.896168 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.896136    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr" event={"ID":"edebe537-c2f9-4728-ab48-98c80d319e25","Type":"ContainerStarted","Data":"c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f"}
Jul 10 03:30:00.896268 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.896194    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr" event={"ID":"edebe537-c2f9-4728-ab48-98c80d319e25","Type":"ContainerStarted","Data":"329b1097614741654435eb525ce76e2b1fcaaba4edc6360eb43c04444a865625"}
Jul 10 03:30:00.910389 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.910344    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t" podStartSLOduration=0.910330357 podStartE2EDuration="910.330357ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:30:00.908857205 +0000 UTC m=+3010.718122507" watchObservedRunningTime="2026-07-10 03:30:00.910330357 +0000 UTC m=+3010.719595659"
Jul 10 03:30:00.923536 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:00.923492    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr" podStartSLOduration=0.92347793 podStartE2EDuration="923.47793ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:30:00.921670986 +0000 UTC m=+3010.730936301" watchObservedRunningTime="2026-07-10 03:30:00.92347793 +0000 UTC m=+3010.732743289"
Jul 10 03:30:49.553827 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:49.551129    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"]
Jul 10 03:30:49.553827 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:49.551462    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy" containerID="cri-o://37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492" gracePeriod=30
Jul 10 03:30:54.610104 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.610066    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 10 03:30:54.610472 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.610133    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 10 03:30:54.812222 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.812197    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:30:54.912416 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912325    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-data\") pod \"3bd84587-f900-4a26-9f3e-f0462f953f40\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") "
Jul 10 03:30:54.912416 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912369    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-envoy\") pod \"3bd84587-f900-4a26-9f3e-f0462f953f40\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") "
Jul 10 03:30:54.912416 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912399    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-token\") pod \"3bd84587-f900-4a26-9f3e-f0462f953f40\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") "
Jul 10 03:30:54.912782 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912452    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgq5j\" (UniqueName: \"kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-kube-api-access-kgq5j\") pod \"3bd84587-f900-4a26-9f3e-f0462f953f40\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") "
Jul 10 03:30:54.912782 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912513    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-certs\") pod \"3bd84587-f900-4a26-9f3e-f0462f953f40\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") "
Jul 10 03:30:54.912782 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912535    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-socket\") pod \"3bd84587-f900-4a26-9f3e-f0462f953f40\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") "
Jul 10 03:30:54.912782 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912552    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-podinfo\") pod \"3bd84587-f900-4a26-9f3e-f0462f953f40\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") "
Jul 10 03:30:54.912782 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912586    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-credential-socket\") pod \"3bd84587-f900-4a26-9f3e-f0462f953f40\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") "
Jul 10 03:30:54.912782 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912618    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd84587-f900-4a26-9f3e-f0462f953f40-istiod-ca-cert\") pod \"3bd84587-f900-4a26-9f3e-f0462f953f40\" (UID: \"3bd84587-f900-4a26-9f3e-f0462f953f40\") "
Jul 10 03:30:54.913110 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912865    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3bd84587-f900-4a26-9f3e-f0462f953f40" (UID: "3bd84587-f900-4a26-9f3e-f0462f953f40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:30:54.913110 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.912998    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:30:54.913217 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.913104    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3bd84587-f900-4a26-9f3e-f0462f953f40" (UID: "3bd84587-f900-4a26-9f3e-f0462f953f40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:30:54.913217 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.913179    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bd84587-f900-4a26-9f3e-f0462f953f40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3bd84587-f900-4a26-9f3e-f0462f953f40" (UID: "3bd84587-f900-4a26-9f3e-f0462f953f40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:30:54.913217 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.913209    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3bd84587-f900-4a26-9f3e-f0462f953f40" (UID: "3bd84587-f900-4a26-9f3e-f0462f953f40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:30:54.913385 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.913367    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-data" (OuterVolumeSpecName: "istio-data") pod "3bd84587-f900-4a26-9f3e-f0462f953f40" (UID: "3bd84587-f900-4a26-9f3e-f0462f953f40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:30:54.914803 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.914775    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-kube-api-access-kgq5j" (OuterVolumeSpecName: "kube-api-access-kgq5j") pod "3bd84587-f900-4a26-9f3e-f0462f953f40" (UID: "3bd84587-f900-4a26-9f3e-f0462f953f40"). InnerVolumeSpecName "kube-api-access-kgq5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:30:54.914918 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.914901    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3bd84587-f900-4a26-9f3e-f0462f953f40" (UID: "3bd84587-f900-4a26-9f3e-f0462f953f40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:30:54.915051 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.915036    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3bd84587-f900-4a26-9f3e-f0462f953f40" (UID: "3bd84587-f900-4a26-9f3e-f0462f953f40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:30:54.915096 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:54.915074    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-token" (OuterVolumeSpecName: "istio-token") pod "3bd84587-f900-4a26-9f3e-f0462f953f40" (UID: "3bd84587-f900-4a26-9f3e-f0462f953f40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:30:55.014241 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.014198    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgq5j\" (UniqueName: \"kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-kube-api-access-kgq5j\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:30:55.014241 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.014232    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:30:55.014241 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.014240    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:30:55.014241 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.014249    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:30:55.014507 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.014257    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd84587-f900-4a26-9f3e-f0462f953f40-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:30:55.014507 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.014265    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:30:55.014507 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.014272    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:30:55.014507 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.014279    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd84587-f900-4a26-9f3e-f0462f953f40-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:30:55.185594 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.185488    2576 generic.go:356] "Generic (PLEG): container finished" podID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerID="37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492" exitCode=0
Jul 10 03:30:55.185594 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.185557    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"
Jul 10 03:30:55.185594 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.185577    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" event={"ID":"3bd84587-f900-4a26-9f3e-f0462f953f40","Type":"ContainerDied","Data":"37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492"}
Jul 10 03:30:55.185890 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.185673    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7" event={"ID":"3bd84587-f900-4a26-9f3e-f0462f953f40","Type":"ContainerDied","Data":"7055f318c595c0a13bb1788d77cccaf61c99800c431fd5c319341c346c45f59a"}
Jul 10 03:30:55.185890 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.185698    2576 scope.go:117] "RemoveContainer" containerID="37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492"
Jul 10 03:30:55.196317 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.196300    2576 scope.go:117] "RemoveContainer" containerID="e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12"
Jul 10 03:30:55.205905 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.205881    2576 scope.go:117] "RemoveContainer" containerID="37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492"
Jul 10 03:30:55.206239 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:30:55.206216    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492\": container with ID starting with 37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492 not found: ID does not exist" containerID="37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492"
Jul 10 03:30:55.206323 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.206247    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492"} err="failed to get container status \"37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492\": rpc error: code = NotFound desc = could not find container \"37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492\": container with ID starting with 37c1ad2397e74f47c3bbcbc0f766d587e86fc88fbbff5b1c63dae3b771272492 not found: ID does not exist"
Jul 10 03:30:55.206323 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.206266    2576 scope.go:117] "RemoveContainer" containerID="e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12"
Jul 10 03:30:55.206593 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:30:55.206566    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12\": container with ID starting with e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12 not found: ID does not exist" containerID="e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12"
Jul 10 03:30:55.206639 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.206604    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12"} err="failed to get container status \"e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12\": rpc error: code = NotFound desc = could not find container \"e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12\": container with ID starting with e00085adb2b807f1ee7b67e5376fdb02d2479a22bb895b816abed56e8f4c5c12 not found: ID does not exist"
Jul 10 03:30:55.207876 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.207856    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"]
Jul 10 03:30:55.211635 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:55.211617    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e7f792-openshift-default-858c87f9d9-xvkv7"]
Jul 10 03:30:56.799792 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:30:56.799727    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" path="/var/lib/kubelet/pods/3bd84587-f900-4a26-9f3e-f0462f953f40/volumes"
Jul 10 03:31:00.009843 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:00.009801    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr"]
Jul 10 03:31:00.010392 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:00.010047    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr" podUID="edebe537-c2f9-4728-ab48-98c80d319e25" containerName="cleanup" containerID="cri-o://c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f" gracePeriod=30
Jul 10 03:31:00.014298 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:00.014276    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t"]
Jul 10 03:31:00.014468 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:00.014445    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t" podUID="9667e21b-36ff-4920-a149-a3ed9d30d5a3" containerName="cleanup" containerID="cri-o://c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf" gracePeriod=30
Jul 10 03:31:30.204575 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.204550    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t"
Jul 10 03:31:30.230504 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.230477    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr"
Jul 10 03:31:30.372634 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.372536    2576 generic.go:356] "Generic (PLEG): container finished" podID="9667e21b-36ff-4920-a149-a3ed9d30d5a3" containerID="c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf" exitCode=137
Jul 10 03:31:30.372634 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.372583    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t"
Jul 10 03:31:30.372634 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.372618    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t" event={"ID":"9667e21b-36ff-4920-a149-a3ed9d30d5a3","Type":"ContainerDied","Data":"c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf"}
Jul 10 03:31:30.372944 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.372675    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t" event={"ID":"9667e21b-36ff-4920-a149-a3ed9d30d5a3","Type":"ContainerDied","Data":"ce0dfd3ff37bdf0c95d2ca8ad90b2336eeb63bb5f4b54905554092586a46a455"}
Jul 10 03:31:30.372944 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.372707    2576 scope.go:117] "RemoveContainer" containerID="c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf"
Jul 10 03:31:30.373988 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.373966    2576 generic.go:356] "Generic (PLEG): container finished" podID="edebe537-c2f9-4728-ab48-98c80d319e25" containerID="c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f" exitCode=137
Jul 10 03:31:30.374111 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.373998    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr" event={"ID":"edebe537-c2f9-4728-ab48-98c80d319e25","Type":"ContainerDied","Data":"c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f"}
Jul 10 03:31:30.374111 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.374025    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr"
Jul 10 03:31:30.374111 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.374049    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr" event={"ID":"edebe537-c2f9-4728-ab48-98c80d319e25","Type":"ContainerDied","Data":"329b1097614741654435eb525ce76e2b1fcaaba4edc6360eb43c04444a865625"}
Jul 10 03:31:30.384885 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.384846    2576 scope.go:117] "RemoveContainer" containerID="c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf"
Jul 10 03:31:30.385199 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:31:30.385178    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf\": container with ID starting with c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf not found: ID does not exist" containerID="c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf"
Jul 10 03:31:30.385295 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.385210    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf"} err="failed to get container status \"c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf\": rpc error: code = NotFound desc = could not find container \"c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf\": container with ID starting with c30e03aac570304842b16b9c34d9139b359c1de9baaf6ac4428f1929f52e39bf not found: ID does not exist"
Jul 10 03:31:30.385295 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.385234    2576 scope.go:117] "RemoveContainer" containerID="c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f"
Jul 10 03:31:30.393072 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.393048    2576 scope.go:117] "RemoveContainer" containerID="c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f"
Jul 10 03:31:30.393331 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:31:30.393313    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f\": container with ID starting with c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f not found: ID does not exist" containerID="c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f"
Jul 10 03:31:30.393381 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.393340    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f"} err="failed to get container status \"c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f\": rpc error: code = NotFound desc = could not find container \"c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f\": container with ID starting with c776dd8f4c96b986c446c34054f1424b46ef718feb13066fc1246c33f3433a1f not found: ID does not exist"
Jul 10 03:31:30.399874 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.399853    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t"]
Jul 10 03:31:30.403437 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.403414    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727570-kpd7t"]
Jul 10 03:31:30.412233 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.412211    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr"]
Jul 10 03:31:30.416599 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.416576    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-48f01f-29727570-dt2rr"]
Jul 10 03:31:30.799072 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.799039    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9667e21b-36ff-4920-a149-a3ed9d30d5a3" path="/var/lib/kubelet/pods/9667e21b-36ff-4920-a149-a3ed9d30d5a3/volumes"
Jul 10 03:31:30.799305 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:31:30.799292    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edebe537-c2f9-4728-ab48-98c80d319e25" path="/var/lib/kubelet/pods/edebe537-c2f9-4728-ab48-98c80d319e25/volumes"
Jul 10 03:32:02.176600 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:02.176566    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"]
Jul 10 03:32:02.177238 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:02.176838    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy" containerID="cri-o://215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34" gracePeriod=30
Jul 10 03:32:03.131736 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.131702    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"]
Jul 10 03:32:03.132136 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132123    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edebe537-c2f9-4728-ab48-98c80d319e25" containerName="cleanup"
Jul 10 03:32:03.132183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132138    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="edebe537-c2f9-4728-ab48-98c80d319e25" containerName="cleanup"
Jul 10 03:32:03.132183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132152    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy"
Jul 10 03:32:03.132183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132157    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy"
Jul 10 03:32:03.132183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132163    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy"
Jul 10 03:32:03.132183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132169    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy"
Jul 10 03:32:03.132183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132175    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9667e21b-36ff-4920-a149-a3ed9d30d5a3" containerName="cleanup"
Jul 10 03:32:03.132183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132181    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9667e21b-36ff-4920-a149-a3ed9d30d5a3" containerName="cleanup"
Jul 10 03:32:03.132393 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132255    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy"
Jul 10 03:32:03.132393 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132263    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="edebe537-c2f9-4728-ab48-98c80d319e25" containerName="cleanup"
Jul 10 03:32:03.132393 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.132270    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9667e21b-36ff-4920-a149-a3ed9d30d5a3" containerName="cleanup"
Jul 10 03:32:03.135436 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.135417    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.137893 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.137871    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-1bdfdf-openshift-default-dockercfg-5vlvn\""
Jul 10 03:32:03.145908 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.145885    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"]
Jul 10 03:32:03.283917 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.283870    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-socket\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.284288 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.283924    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0127b53-9348-4665-aa0d-270433782fcf-istio-podinfo\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.284288 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.284019    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-envoy\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.284288 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.284064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-istio-token\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.284288 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.284092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwqc2\" (UniqueName: \"kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-kube-api-access-wwqc2\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.284288 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.284146    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-data\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.284288 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.284197    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-certs\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.284288 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.284215    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-credential-socket\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.284288 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.284244    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0127b53-9348-4665-aa0d-270433782fcf-istiod-ca-cert\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.385308 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385201    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-socket\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.385308 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385245    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0127b53-9348-4665-aa0d-270433782fcf-istio-podinfo\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.385308 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-envoy\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.385600 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-istio-token\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.385600 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwqc2\" (UniqueName: \"kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-kube-api-access-wwqc2\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.385600 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385410    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-data\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.385600 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-certs\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.385600 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385476    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-credential-socket\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.385600 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385515    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0127b53-9348-4665-aa0d-270433782fcf-istiod-ca-cert\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.385962 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-socket\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.386034 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.385978    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-certs\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.386310 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.386284    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0127b53-9348-4665-aa0d-270433782fcf-istiod-ca-cert\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.386458 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.386412    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-data\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.386947 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.386889    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-credential-socket\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.389259 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.389238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-envoy\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.389580 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.389559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0127b53-9348-4665-aa0d-270433782fcf-istio-podinfo\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.394185 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.394156    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwqc2\" (UniqueName: \"kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-kube-api-access-wwqc2\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.394267 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.394234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-istio-token\") pod \"e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") " pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.446646 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.446614    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:03.583137 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.583108    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"]
Jul 10 03:32:03.584954 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:32:03.584923    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0127b53_9348_4665_aa0d_270433782fcf.slice/crio-2a3928dd9375504f3a7c2cfd81b9a6e9e8c9803edc038b34fa55ff2368bd617d WatchSource:0}: Error finding container 2a3928dd9375504f3a7c2cfd81b9a6e9e8c9803edc038b34fa55ff2368bd617d: Status 404 returned error can't find the container with id 2a3928dd9375504f3a7c2cfd81b9a6e9e8c9803edc038b34fa55ff2368bd617d
Jul 10 03:32:03.587006 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.586959    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:03.587113 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.587043    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:03.587113 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:03.587088    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:04.555365 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:04.555333    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" event={"ID":"e0127b53-9348-4665-aa0d-270433782fcf","Type":"ContainerStarted","Data":"61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e"}
Jul 10 03:32:04.555733 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:04.555388    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" event={"ID":"e0127b53-9348-4665-aa0d-270433782fcf","Type":"ContainerStarted","Data":"2a3928dd9375504f3a7c2cfd81b9a6e9e8c9803edc038b34fa55ff2368bd617d"}
Jul 10 03:32:04.574627 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:04.574586    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" podStartSLOduration=1.574571256 podStartE2EDuration="1.574571256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:04.57363074 +0000 UTC m=+3134.382896055" watchObservedRunningTime="2026-07-10 03:32:04.574571256 +0000 UTC m=+3134.383836557"
Jul 10 03:32:05.447543 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:05.447507    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:05.452543 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:05.452508    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:05.567064 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:05.567033    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:05.568007 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:05.567991    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:07.430238 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.430180    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:32:07.526168 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526140    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-envoy\") pod \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") "
Jul 10 03:32:07.526346 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526189    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkxp9\" (UniqueName: \"kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-kube-api-access-jkxp9\") pod \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") "
Jul 10 03:32:07.526346 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526208    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-credential-socket\") pod \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") "
Jul 10 03:32:07.526346 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526225    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-certs\") pod \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") "
Jul 10 03:32:07.526346 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526272    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-token\") pod \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") "
Jul 10 03:32:07.526346 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526307    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-podinfo\") pod \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") "
Jul 10 03:32:07.526597 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526349    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istiod-ca-cert\") pod \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") "
Jul 10 03:32:07.526597 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526390    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-socket\") pod \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") "
Jul 10 03:32:07.526597 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526420    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-data\") pod \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\" (UID: \"8498a3bb-ed2f-487a-8648-1eba8d08fd24\") "
Jul 10 03:32:07.526597 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526485    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8498a3bb-ed2f-487a-8648-1eba8d08fd24" (UID: "8498a3bb-ed2f-487a-8648-1eba8d08fd24"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:32:07.526854 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526644    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8498a3bb-ed2f-487a-8648-1eba8d08fd24" (UID: "8498a3bb-ed2f-487a-8648-1eba8d08fd24"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:32:07.526854 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526842    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:32:07.526958 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526864    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:32:07.526958 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526905    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8498a3bb-ed2f-487a-8648-1eba8d08fd24" (UID: "8498a3bb-ed2f-487a-8648-1eba8d08fd24"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:32:07.527049 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.526947    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8498a3bb-ed2f-487a-8648-1eba8d08fd24" (UID: "8498a3bb-ed2f-487a-8648-1eba8d08fd24"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:32:07.527480 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.527456    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-data" (OuterVolumeSpecName: "istio-data") pod "8498a3bb-ed2f-487a-8648-1eba8d08fd24" (UID: "8498a3bb-ed2f-487a-8648-1eba8d08fd24"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:32:07.528507 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.528481    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-token" (OuterVolumeSpecName: "istio-token") pod "8498a3bb-ed2f-487a-8648-1eba8d08fd24" (UID: "8498a3bb-ed2f-487a-8648-1eba8d08fd24"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:32:07.528639 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.528619    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8498a3bb-ed2f-487a-8648-1eba8d08fd24" (UID: "8498a3bb-ed2f-487a-8648-1eba8d08fd24"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:32:07.528812 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.528781    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8498a3bb-ed2f-487a-8648-1eba8d08fd24" (UID: "8498a3bb-ed2f-487a-8648-1eba8d08fd24"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:32:07.528812 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.528784    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-kube-api-access-jkxp9" (OuterVolumeSpecName: "kube-api-access-jkxp9") pod "8498a3bb-ed2f-487a-8648-1eba8d08fd24" (UID: "8498a3bb-ed2f-487a-8648-1eba8d08fd24"). InnerVolumeSpecName "kube-api-access-jkxp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:32:07.577970 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.577936    2576 generic.go:356] "Generic (PLEG): container finished" podID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerID="215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34" exitCode=0
Jul 10 03:32:07.578131 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.577980    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"
Jul 10 03:32:07.578131 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.578020    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" event={"ID":"8498a3bb-ed2f-487a-8648-1eba8d08fd24","Type":"ContainerDied","Data":"215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34"}
Jul 10 03:32:07.578131 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.578069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq" event={"ID":"8498a3bb-ed2f-487a-8648-1eba8d08fd24","Type":"ContainerDied","Data":"2774ad9fa20cadce1bf06e5b157bfd826f743c8ce8debedad4c46b0613355ad7"}
Jul 10 03:32:07.578131 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.578092    2576 scope.go:117] "RemoveContainer" containerID="215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34"
Jul 10 03:32:07.587238 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.587223    2576 scope.go:117] "RemoveContainer" containerID="11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a"
Jul 10 03:32:07.595896 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.595876    2576 scope.go:117] "RemoveContainer" containerID="215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34"
Jul 10 03:32:07.596150 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:32:07.596131    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34\": container with ID starting with 215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34 not found: ID does not exist" containerID="215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34"
Jul 10 03:32:07.596202 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.596158    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34"} err="failed to get container status \"215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34\": rpc error: code = NotFound desc = could not find container \"215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34\": container with ID starting with 215e1e3ac7a20c91b3f822f1ad8558cca5f529f3af8da63131178b1a55b30f34 not found: ID does not exist"
Jul 10 03:32:07.596202 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.596175    2576 scope.go:117] "RemoveContainer" containerID="11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a"
Jul 10 03:32:07.596387 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:32:07.596369    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a\": container with ID starting with 11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a not found: ID does not exist" containerID="11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a"
Jul 10 03:32:07.596429 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.596392    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a"} err="failed to get container status \"11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a\": rpc error: code = NotFound desc = could not find container \"11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a\": container with ID starting with 11d1670e35a406ab30d45636006309145ae2b1c82a61eaefac77172c08d7980a not found: ID does not exist"
Jul 10 03:32:07.602854 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.602829    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"]
Jul 10 03:32:07.608267 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.608247    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-48f01f-openshift-default-5cf77466d5-h66wq"]
Jul 10 03:32:07.627405 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.627379    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:32:07.627545 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.627410    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:32:07.627545 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.627424    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:32:07.627545 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.627436    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:32:07.627545 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.627447    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:32:07.627545 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.627544    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8498a3bb-ed2f-487a-8648-1eba8d08fd24-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:32:07.627791 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:07.627559    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkxp9\" (UniqueName: \"kubernetes.io/projected/8498a3bb-ed2f-487a-8648-1eba8d08fd24-kube-api-access-jkxp9\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:32:08.799515 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:08.799478    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" path="/var/lib/kubelet/pods/8498a3bb-ed2f-487a-8648-1eba8d08fd24/volumes"
Jul 10 03:32:11.868166 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.868137    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"]
Jul 10 03:32:11.868576 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.868562    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy"
Jul 10 03:32:11.868576 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.868576    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy"
Jul 10 03:32:11.868682 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.868586    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy"
Jul 10 03:32:11.868682 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.868592    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy"
Jul 10 03:32:11.868682 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.868661    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd84587-f900-4a26-9f3e-f0462f953f40" containerName="istio-proxy"
Jul 10 03:32:11.868682 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.868670    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy"
Jul 10 03:32:11.868682 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.868676    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8498a3bb-ed2f-487a-8648-1eba8d08fd24" containerName="istio-proxy"
Jul 10 03:32:11.873107 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.873084    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:11.875364 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.875343    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9d01ec-openshift-default-dockercfg-v6jn8\""
Jul 10 03:32:11.883961 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.883933    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"]
Jul 10 03:32:11.967689 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.967654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-socket\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:11.967689 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.967690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-envoy\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:11.967970 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.967790    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04138795-7174-48b1-971a-5419446a1644-istio-podinfo\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:11.967970 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.967850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04138795-7174-48b1-971a-5419446a1644-istiod-ca-cert\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:11.967970 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.967935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-certs\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:11.968148 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.967969    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-istio-token\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:11.968148 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.968111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm4tg\" (UniqueName: \"kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-kube-api-access-pm4tg\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:11.968263 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.968155    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-data\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:11.968263 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:11.968196    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-credential-socket\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069206 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm4tg\" (UniqueName: \"kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-kube-api-access-pm4tg\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069206 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-data\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069438 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-credential-socket\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069438 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069267    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-socket\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069438 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-envoy\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069438 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069342    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04138795-7174-48b1-971a-5419446a1644-istio-podinfo\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069438 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04138795-7174-48b1-971a-5419446a1644-istiod-ca-cert\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069438 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-certs\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069790 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-istio-token\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069863 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069786    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-credential-socket\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069863 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-socket\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.069972 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.069937    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-data\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.070090 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.070065    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-certs\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.070213 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.070192    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04138795-7174-48b1-971a-5419446a1644-istiod-ca-cert\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.071803 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.071782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-envoy\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.072036 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.072018    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04138795-7174-48b1-971a-5419446a1644-istio-podinfo\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.077030 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.077005    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-istio-token\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.077263 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.077243    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm4tg\" (UniqueName: \"kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-kube-api-access-pm4tg\") pod \"e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") " pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.185942 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.185825    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:12.374769 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.374721    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"]
Jul 10 03:32:12.376426 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:32:12.376396    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04138795_7174_48b1_971a_5419446a1644.slice/crio-1d53c88cccc42b9578fe531e0e5ee267bbd73269b5ca9312ebc6f2a75bc12e25 WatchSource:0}: Error finding container 1d53c88cccc42b9578fe531e0e5ee267bbd73269b5ca9312ebc6f2a75bc12e25: Status 404 returned error can't find the container with id 1d53c88cccc42b9578fe531e0e5ee267bbd73269b5ca9312ebc6f2a75bc12e25
Jul 10 03:32:12.378688 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.378656    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:12.378783 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.378719    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:12.378783 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.378762    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:12.607429 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.607392    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" event={"ID":"04138795-7174-48b1-971a-5419446a1644","Type":"ContainerStarted","Data":"f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57"}
Jul 10 03:32:12.607632 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.607447    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" event={"ID":"04138795-7174-48b1-971a-5419446a1644","Type":"ContainerStarted","Data":"1d53c88cccc42b9578fe531e0e5ee267bbd73269b5ca9312ebc6f2a75bc12e25"}
Jul 10 03:32:12.627317 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:12.627256    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" podStartSLOduration=1.6272327610000001 podStartE2EDuration="1.627232761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:12.625458615 +0000 UTC m=+3142.434723915" watchObservedRunningTime="2026-07-10 03:32:12.627232761 +0000 UTC m=+3142.436498065"
Jul 10 03:32:13.186713 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:13.186677    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:13.191864 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:13.191838    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:13.612820 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:13.612786    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:13.613815 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:13.613798    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:19.670123 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:19.670033    2576 generic.go:356] "Generic (PLEG): container finished" podID="e0127b53-9348-4665-aa0d-270433782fcf" containerID="61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e" exitCode=0
Jul 10 03:32:19.670123 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:19.670097    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" event={"ID":"e0127b53-9348-4665-aa0d-270433782fcf","Type":"ContainerDied","Data":"61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e"}
Jul 10 03:32:19.670559 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:19.670483    2576 scope.go:117] "RemoveContainer" containerID="61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e"
Jul 10 03:32:19.671513 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:19.671486    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:19.671563 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:19.671553    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:19.671599 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:19.671584    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:20.567427 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:20.567391    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:20.676661 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:20.676621    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" event={"ID":"e0127b53-9348-4665-aa0d-270433782fcf","Type":"ContainerStarted","Data":"d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1"}
Jul 10 03:32:21.447423 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:21.447378    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:21.448673 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:21.448646    2576 patch_prober.go:28] interesting pod/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 10 03:32:21.448825 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:21.448694    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 10 03:32:22.447313 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:22.447274    2576 patch_prober.go:28] interesting pod/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 10 03:32:22.447681 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:22.447348    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 10 03:32:23.452771 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:23.447696    2576 patch_prober.go:28] interesting pod/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 10 03:32:23.452771 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:23.447783    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 10 03:32:24.449492 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:24.449455    2576 patch_prober.go:28] interesting pod/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 10 03:32:24.449810 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:24.449779    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 10 03:32:25.451872 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:25.451842    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:25.452304 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:25.452118    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:25.452834 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:25.452817    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:32:29.654986 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:29.654937    2576 patch_prober.go:28] interesting pod/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 03:32:29.655387 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:29.655029    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 03:32:30.732984 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:30.732950    2576 generic.go:356] "Generic (PLEG): container finished" podID="04138795-7174-48b1-971a-5419446a1644" containerID="f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57" exitCode=0
Jul 10 03:32:30.733429 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:30.733022    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" event={"ID":"04138795-7174-48b1-971a-5419446a1644","Type":"ContainerDied","Data":"f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57"}
Jul 10 03:32:30.733489 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:30.733460    2576 scope.go:117] "RemoveContainer" containerID="f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57"
Jul 10 03:32:30.734521 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:30.734496    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:30.734578 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:30.734568    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:30.734614 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:30.734600    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:32:31.743663 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:31.743615    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" event={"ID":"04138795-7174-48b1-971a-5419446a1644","Type":"ContainerStarted","Data":"26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6"}
Jul 10 03:32:32.186353 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:32.186251    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:32.186626 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:32.186517    2576 patch_prober.go:28] interesting pod/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 03:32:32.186626 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:32.186563    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 03:32:33.186607 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:33.186565    2576 patch_prober.go:28] interesting pod/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 03:32:33.187109 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:33.186642    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 03:32:34.186861 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:34.186819    2576 patch_prober.go:28] interesting pod/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 03:32:34.187340 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:34.186886    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 03:32:35.187082 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:35.187048    2576 patch_prober.go:28] interesting pod/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 03:32:35.187463 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:35.187133    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 03:32:36.190026 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:36.189991    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:36.190511 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:36.190398    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:32:36.191163 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:32:36.191145    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:33:39.288472 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:39.288432    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"]
Jul 10 03:33:39.289095 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:39.288698    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy" containerID="cri-o://d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1" gracePeriod=30
Jul 10 03:33:40.454901 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:40.454867    2576 patch_prober.go:28] interesting pod/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 03:33:40.455284 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:40.454923    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 03:33:44.547781 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.547736    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:33:44.700932 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.700832    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-envoy\") pod \"e0127b53-9348-4665-aa0d-270433782fcf\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") "
Jul 10 03:33:44.700932 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.700883    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0127b53-9348-4665-aa0d-270433782fcf-istiod-ca-cert\") pod \"e0127b53-9348-4665-aa0d-270433782fcf\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") "
Jul 10 03:33:44.700932 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.700908    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-credential-socket\") pod \"e0127b53-9348-4665-aa0d-270433782fcf\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") "
Jul 10 03:33:44.701328 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.700937    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-data\") pod \"e0127b53-9348-4665-aa0d-270433782fcf\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") "
Jul 10 03:33:44.701328 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.700990    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwqc2\" (UniqueName: \"kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-kube-api-access-wwqc2\") pod \"e0127b53-9348-4665-aa0d-270433782fcf\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") "
Jul 10 03:33:44.701328 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701014    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-socket\") pod \"e0127b53-9348-4665-aa0d-270433782fcf\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") "
Jul 10 03:33:44.701328 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701119    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0127b53-9348-4665-aa0d-270433782fcf-istio-podinfo\") pod \"e0127b53-9348-4665-aa0d-270433782fcf\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") "
Jul 10 03:33:44.701328 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701151    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-istio-token\") pod \"e0127b53-9348-4665-aa0d-270433782fcf\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") "
Jul 10 03:33:44.701328 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701178    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-certs\") pod \"e0127b53-9348-4665-aa0d-270433782fcf\" (UID: \"e0127b53-9348-4665-aa0d-270433782fcf\") "
Jul 10 03:33:44.701626 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701291    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0127b53-9348-4665-aa0d-270433782fcf" (UID: "e0127b53-9348-4665-aa0d-270433782fcf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:33:44.701626 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701370    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0127b53-9348-4665-aa0d-270433782fcf" (UID: "e0127b53-9348-4665-aa0d-270433782fcf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:33:44.701626 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701376    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0127b53-9348-4665-aa0d-270433782fcf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0127b53-9348-4665-aa0d-270433782fcf" (UID: "e0127b53-9348-4665-aa0d-270433782fcf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:33:44.701626 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701574    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0127b53-9348-4665-aa0d-270433782fcf" (UID: "e0127b53-9348-4665-aa0d-270433782fcf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:33:44.701626 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701590    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:33:44.701626 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701613    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:33:44.701626 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.701627    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0127b53-9348-4665-aa0d-270433782fcf-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:33:44.702041 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.702005    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0127b53-9348-4665-aa0d-270433782fcf" (UID: "e0127b53-9348-4665-aa0d-270433782fcf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:33:44.703333 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.703305    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0127b53-9348-4665-aa0d-270433782fcf" (UID: "e0127b53-9348-4665-aa0d-270433782fcf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:33:44.703333 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.703313    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0127b53-9348-4665-aa0d-270433782fcf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0127b53-9348-4665-aa0d-270433782fcf" (UID: "e0127b53-9348-4665-aa0d-270433782fcf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:33:44.703527 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.703449    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-kube-api-access-wwqc2" (OuterVolumeSpecName: "kube-api-access-wwqc2") pod "e0127b53-9348-4665-aa0d-270433782fcf" (UID: "e0127b53-9348-4665-aa0d-270433782fcf"). InnerVolumeSpecName "kube-api-access-wwqc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:33:44.703527 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.703462    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0127b53-9348-4665-aa0d-270433782fcf" (UID: "e0127b53-9348-4665-aa0d-270433782fcf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:33:44.803041 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.802883    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0127b53-9348-4665-aa0d-270433782fcf-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:33:44.803041 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.802919    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:33:44.803041 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.802931    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:33:44.803041 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.802944    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:33:44.803041 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.802962    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0127b53-9348-4665-aa0d-270433782fcf-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:33:44.803041 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:44.802974    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwqc2\" (UniqueName: \"kubernetes.io/projected/e0127b53-9348-4665-aa0d-270433782fcf-kube-api-access-wwqc2\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:33:45.128959 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.128922    2576 generic.go:356] "Generic (PLEG): container finished" podID="e0127b53-9348-4665-aa0d-270433782fcf" containerID="d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1" exitCode=0
Jul 10 03:33:45.129174 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.128989    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"
Jul 10 03:33:45.129174 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.129000    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" event={"ID":"e0127b53-9348-4665-aa0d-270433782fcf","Type":"ContainerDied","Data":"d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1"}
Jul 10 03:33:45.129174 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.129057    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4" event={"ID":"e0127b53-9348-4665-aa0d-270433782fcf","Type":"ContainerDied","Data":"2a3928dd9375504f3a7c2cfd81b9a6e9e8c9803edc038b34fa55ff2368bd617d"}
Jul 10 03:33:45.129174 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.129087    2576 scope.go:117] "RemoveContainer" containerID="d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1"
Jul 10 03:33:45.138188 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.138171    2576 scope.go:117] "RemoveContainer" containerID="61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e"
Jul 10 03:33:45.147077 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.147057    2576 scope.go:117] "RemoveContainer" containerID="d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1"
Jul 10 03:33:45.147242 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.147215    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"]
Jul 10 03:33:45.147366 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:33:45.147346    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1\": container with ID starting with d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1 not found: ID does not exist" containerID="d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1"
Jul 10 03:33:45.147414 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.147374    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1"} err="failed to get container status \"d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1\": rpc error: code = NotFound desc = could not find container \"d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1\": container with ID starting with d28fd8e95100c7f82b42fa0ef679662f2a5776783f9c2334ac05aa97e53446b1 not found: ID does not exist"
Jul 10 03:33:45.147414 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.147391    2576 scope.go:117] "RemoveContainer" containerID="61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e"
Jul 10 03:33:45.147659 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:33:45.147640    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e\": container with ID starting with 61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e not found: ID does not exist" containerID="61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e"
Jul 10 03:33:45.147711 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.147668    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e"} err="failed to get container status \"61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e\": rpc error: code = NotFound desc = could not find container \"61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e\": container with ID starting with 61d0acac9a3b0975ec0d02ca895c0bc938615c7a3448424ad9b4e2836abb7d6e not found: ID does not exist"
Jul 10 03:33:45.150978 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:45.150954    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-1bdfdf-openshift-default-6787f6b8d9-6jhp4"]
Jul 10 03:33:46.798831 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:33:46.798796    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0127b53-9348-4665-aa0d-270433782fcf" path="/var/lib/kubelet/pods/e0127b53-9348-4665-aa0d-270433782fcf/volumes"
Jul 10 03:34:50.928398 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:50.928371    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:34:50.934689 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:50.934666    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:34:51.729733 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:51.729696    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"]
Jul 10 03:34:51.730041 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:51.729996    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy" containerID="cri-o://26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6" gracePeriod=30
Jul 10 03:34:52.695199 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.695165    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"]
Jul 10 03:34:52.695793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.695771    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy"
Jul 10 03:34:52.695793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.695795    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy"
Jul 10 03:34:52.695957 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.695932    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy"
Jul 10 03:34:52.695957 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.695946    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy"
Jul 10 03:34:52.699305 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.699282    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.701956 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.701935    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a7b338-openshift-default-dockercfg-zb6rw\""
Jul 10 03:34:52.712084 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.712061    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"]
Jul 10 03:34:52.784322 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.784274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwgf5\" (UniqueName: \"kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-kube-api-access-kwgf5\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.784529 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.784335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-token\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.784529 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.784388    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-envoy\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.784529 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.784443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-data\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.784729 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.784528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-socket\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.784729 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.784608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istiod-ca-cert\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.784729 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.784644    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-credential-socket\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.784729 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.784672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-podinfo\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.784729 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.784701    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-certs\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.885920 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.885880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istiod-ca-cert\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.886118 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.886092    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-credential-socket\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.886195 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.886139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-podinfo\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.886195 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.886173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-certs\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.886635 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.886368    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwgf5\" (UniqueName: \"kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-kube-api-access-kwgf5\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.886635 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.886425    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-token\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.886635 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.886520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-certs\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.886635 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.886535    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-envoy\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.886635 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.886617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-data\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.887042 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.886666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istiod-ca-cert\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.887042 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.886780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-socket\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.887411 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.887388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-credential-socket\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.887692 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.887652    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-data\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.887824 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.887722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-socket\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.889105 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.889085    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-podinfo\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.889398 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.889380    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-envoy\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.894537 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.894509    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-token\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:52.895154 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:52.895138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwgf5\" (UniqueName: \"kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-kube-api-access-kwgf5\") pod \"e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") " pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:53.009915 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:53.009870    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:53.149882 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:53.149855    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"]
Jul 10 03:34:53.152426 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:34:53.152388    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc8e8c4b_b74d_4af5_a2f9_023639b14d42.slice/crio-1c5ac8b44516dc1dbb00d6faf0e7e97624f5c7571970d999ea354d1232ed7cd2 WatchSource:0}: Error finding container 1c5ac8b44516dc1dbb00d6faf0e7e97624f5c7571970d999ea354d1232ed7cd2: Status 404 returned error can't find the container with id 1c5ac8b44516dc1dbb00d6faf0e7e97624f5c7571970d999ea354d1232ed7cd2
Jul 10 03:34:53.155077 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:53.155041    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:34:53.155187 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:53.155110    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:34:53.155187 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:53.155140    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:34:53.476531 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:53.476492    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" event={"ID":"bc8e8c4b-b74d-4af5-a2f9-023639b14d42","Type":"ContainerStarted","Data":"b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62"}
Jul 10 03:34:53.476737 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:53.476550    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" event={"ID":"bc8e8c4b-b74d-4af5-a2f9-023639b14d42","Type":"ContainerStarted","Data":"1c5ac8b44516dc1dbb00d6faf0e7e97624f5c7571970d999ea354d1232ed7cd2"}
Jul 10 03:34:53.497052 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:53.496998    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" podStartSLOduration=1.496977946 podStartE2EDuration="1.496977946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:34:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:34:53.49439981 +0000 UTC m=+3303.303665126" watchObservedRunningTime="2026-07-10 03:34:53.496977946 +0000 UTC m=+3303.306243248"
Jul 10 03:34:54.011018 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:54.010981    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:54.015959 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:54.015931    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:54.481702 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:54.481675    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:54.482850 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:54.482826    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:34:57.008630 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.008600    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:34:57.136640 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.136603    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-istio-token\") pod \"04138795-7174-48b1-971a-5419446a1644\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") "
Jul 10 03:34:57.136865 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.136665    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-credential-socket\") pod \"04138795-7174-48b1-971a-5419446a1644\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") "
Jul 10 03:34:57.136865 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.136695    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-certs\") pod \"04138795-7174-48b1-971a-5419446a1644\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") "
Jul 10 03:34:57.136865 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.136730    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm4tg\" (UniqueName: \"kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-kube-api-access-pm4tg\") pod \"04138795-7174-48b1-971a-5419446a1644\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") "
Jul 10 03:34:57.136865 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.136787    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04138795-7174-48b1-971a-5419446a1644-istio-podinfo\") pod \"04138795-7174-48b1-971a-5419446a1644\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") "
Jul 10 03:34:57.136865 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.136856    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-envoy\") pod \"04138795-7174-48b1-971a-5419446a1644\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") "
Jul 10 03:34:57.137130 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.136901    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-socket\") pod \"04138795-7174-48b1-971a-5419446a1644\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") "
Jul 10 03:34:57.137130 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.136929    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04138795-7174-48b1-971a-5419446a1644-istiod-ca-cert\") pod \"04138795-7174-48b1-971a-5419446a1644\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") "
Jul 10 03:34:57.137130 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.136975    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-data\") pod \"04138795-7174-48b1-971a-5419446a1644\" (UID: \"04138795-7174-48b1-971a-5419446a1644\") "
Jul 10 03:34:57.137130 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.136979    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04138795-7174-48b1-971a-5419446a1644" (UID: "04138795-7174-48b1-971a-5419446a1644"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:34:57.137130 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.137023    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04138795-7174-48b1-971a-5419446a1644" (UID: "04138795-7174-48b1-971a-5419446a1644"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:34:57.137362 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.137295    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04138795-7174-48b1-971a-5419446a1644-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04138795-7174-48b1-971a-5419446a1644" (UID: "04138795-7174-48b1-971a-5419446a1644"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:34:57.137362 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.137334    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:34:57.137362 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.137353    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:34:57.137508 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.137347    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04138795-7174-48b1-971a-5419446a1644" (UID: "04138795-7174-48b1-971a-5419446a1644"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:34:57.138441 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.138410    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-data" (OuterVolumeSpecName: "istio-data") pod "04138795-7174-48b1-971a-5419446a1644" (UID: "04138795-7174-48b1-971a-5419446a1644"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:34:57.139013 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.138989    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-istio-token" (OuterVolumeSpecName: "istio-token") pod "04138795-7174-48b1-971a-5419446a1644" (UID: "04138795-7174-48b1-971a-5419446a1644"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:34:57.139247 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.139215    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-kube-api-access-pm4tg" (OuterVolumeSpecName: "kube-api-access-pm4tg") pod "04138795-7174-48b1-971a-5419446a1644" (UID: "04138795-7174-48b1-971a-5419446a1644"). InnerVolumeSpecName "kube-api-access-pm4tg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:34:57.139395 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.139378    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04138795-7174-48b1-971a-5419446a1644-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04138795-7174-48b1-971a-5419446a1644" (UID: "04138795-7174-48b1-971a-5419446a1644"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:34:57.139472 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.139448    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04138795-7174-48b1-971a-5419446a1644" (UID: "04138795-7174-48b1-971a-5419446a1644"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:34:57.238209 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.238179    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:34:57.238209 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.238210    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:34:57.238407 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.238224    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04138795-7174-48b1-971a-5419446a1644-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:34:57.238407 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.238233    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04138795-7174-48b1-971a-5419446a1644-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:34:57.238407 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.238241    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:34:57.238407 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.238249    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm4tg\" (UniqueName: \"kubernetes.io/projected/04138795-7174-48b1-971a-5419446a1644-kube-api-access-pm4tg\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:34:57.238407 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.238258    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04138795-7174-48b1-971a-5419446a1644-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:34:57.499171 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.499139    2576 generic.go:356] "Generic (PLEG): container finished" podID="04138795-7174-48b1-971a-5419446a1644" containerID="26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6" exitCode=0
Jul 10 03:34:57.499362 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.499185    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"
Jul 10 03:34:57.499362 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.499218    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" event={"ID":"04138795-7174-48b1-971a-5419446a1644","Type":"ContainerDied","Data":"26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6"}
Jul 10 03:34:57.499362 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.499272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8" event={"ID":"04138795-7174-48b1-971a-5419446a1644","Type":"ContainerDied","Data":"1d53c88cccc42b9578fe531e0e5ee267bbd73269b5ca9312ebc6f2a75bc12e25"}
Jul 10 03:34:57.499362 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.499294    2576 scope.go:117] "RemoveContainer" containerID="26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6"
Jul 10 03:34:57.508691 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.508677    2576 scope.go:117] "RemoveContainer" containerID="f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57"
Jul 10 03:34:57.516690 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.516669    2576 scope.go:117] "RemoveContainer" containerID="26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6"
Jul 10 03:34:57.517011 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:34:57.516992    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6\": container with ID starting with 26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6 not found: ID does not exist" containerID="26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6"
Jul 10 03:34:57.517075 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.517020    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6"} err="failed to get container status \"26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6\": rpc error: code = NotFound desc = could not find container \"26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6\": container with ID starting with 26b1000fc37b2c581f50a4ecdbc33302be9ef831e3e9e92730dc3d3a6f682bc6 not found: ID does not exist"
Jul 10 03:34:57.517075 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.517038    2576 scope.go:117] "RemoveContainer" containerID="f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57"
Jul 10 03:34:57.517335 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:34:57.517314    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57\": container with ID starting with f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57 not found: ID does not exist" containerID="f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57"
Jul 10 03:34:57.517388 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.517340    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57"} err="failed to get container status \"f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57\": rpc error: code = NotFound desc = could not find container \"f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57\": container with ID starting with f91bc48ac8ca354647112dac4f79e5038d24e4a811202622e5d7165523cb2b57 not found: ID does not exist"
Jul 10 03:34:57.520468 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.520446    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"]
Jul 10 03:34:57.526854 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:57.526834    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9d01ec-openshift-default-5cc6649f9c-7fmr8"]
Jul 10 03:34:58.799615 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:34:58.799568    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04138795-7174-48b1-971a-5419446a1644" path="/var/lib/kubelet/pods/04138795-7174-48b1-971a-5419446a1644/volumes"
Jul 10 03:35:00.660439 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.660398    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"]
Jul 10 03:35:00.661071 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.661049    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy"
Jul 10 03:35:00.661071 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.661073    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy"
Jul 10 03:35:00.661266 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.661091    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy"
Jul 10 03:35:00.661266 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.661099    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy"
Jul 10 03:35:00.661266 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.661144    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy"
Jul 10 03:35:00.661266 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.661153    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0127b53-9348-4665-aa0d-270433782fcf" containerName="istio-proxy"
Jul 10 03:35:00.661266 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.661250    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy"
Jul 10 03:35:00.665769 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.665721    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.668164 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.668146    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-17c732-openshift-default-dockercfg-hp7zn\""
Jul 10 03:35:00.670480 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.670460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-envoy\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.670603 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.670492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-istio-token\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.670603 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.670548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58521a5e-8269-4750-bcd4-449df3327cb6-istio-podinfo\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.670603 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.670569    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58521a5e-8269-4750-bcd4-449df3327cb6-istiod-ca-cert\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.670603 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.670586    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-data\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.670859 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.670605    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6lc6\" (UniqueName: \"kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-kube-api-access-k6lc6\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.670859 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.670651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-socket\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.670859 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.670704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-certs\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.670859 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.670723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-credential-socket\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.681087 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.681063    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"]
Jul 10 03:35:00.771625 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.771590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-envoy\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.771831 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.771638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-istio-token\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.771831 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.771692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58521a5e-8269-4750-bcd4-449df3327cb6-istio-podinfo\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.771831 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.771723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58521a5e-8269-4750-bcd4-449df3327cb6-istiod-ca-cert\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.771831 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.771762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-data\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.771831 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.771792    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6lc6\" (UniqueName: \"kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-kube-api-access-k6lc6\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.771831 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.771817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-socket\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.772183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.771869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-certs\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.772183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.771895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-credential-socket\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.772183 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.772151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-data\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.772353 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.772259    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-credential-socket\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.772353 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.772279    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-socket\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.772353 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.772319    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-certs\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.772581 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.772560    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58521a5e-8269-4750-bcd4-449df3327cb6-istiod-ca-cert\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.774038 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.774017    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-envoy\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.774300 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.774282    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58521a5e-8269-4750-bcd4-449df3327cb6-istio-podinfo\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.779341 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.779317    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-istio-token\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.779719 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.779702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6lc6\" (UniqueName: \"kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-kube-api-access-k6lc6\") pod \"e2e-shared-b-17c732-openshift-default-68cb85f679-szm25\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") " pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:00.976197 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:00.976158    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:01.122430 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:01.122397    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"]
Jul 10 03:35:01.125430 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:35:01.125395    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58521a5e_8269_4750_bcd4_449df3327cb6.slice/crio-42538e86257b85196c6623fa91fea90896a61d375b89c847991f16286887fbca WatchSource:0}: Error finding container 42538e86257b85196c6623fa91fea90896a61d375b89c847991f16286887fbca: Status 404 returned error can't find the container with id 42538e86257b85196c6623fa91fea90896a61d375b89c847991f16286887fbca
Jul 10 03:35:01.128085 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:01.128052    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:35:01.128149 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:01.128138    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:35:01.128191 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:01.128166    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:35:01.522995 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:01.522954    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" event={"ID":"58521a5e-8269-4750-bcd4-449df3327cb6","Type":"ContainerStarted","Data":"2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9"}
Jul 10 03:35:01.523172 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:01.523015    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" event={"ID":"58521a5e-8269-4750-bcd4-449df3327cb6","Type":"ContainerStarted","Data":"42538e86257b85196c6623fa91fea90896a61d375b89c847991f16286887fbca"}
Jul 10 03:35:01.542221 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:01.542170    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" podStartSLOduration=1.542153117 podStartE2EDuration="1.542153117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:35:01.539646951 +0000 UTC m=+3311.348912265" watchObservedRunningTime="2026-07-10 03:35:01.542153117 +0000 UTC m=+3311.351418419"
Jul 10 03:35:01.977083 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:01.977045    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:02.982361 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:02.982329    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:03.539889 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:03.539849    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:03.540809 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:03.540779    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:09.483593 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:09.483553    2576 patch_prober.go:28] interesting pod/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 03:35:09.484106 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:09.483617    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 03:35:09.572452 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:09.572415    2576 generic.go:356] "Generic (PLEG): container finished" podID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerID="b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62" exitCode=0
Jul 10 03:35:09.572646 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:09.572484    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" event={"ID":"bc8e8c4b-b74d-4af5-a2f9-023639b14d42","Type":"ContainerDied","Data":"b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62"}
Jul 10 03:35:09.572916 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:09.572898    2576 scope.go:117] "RemoveContainer" containerID="b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62"
Jul 10 03:35:09.573910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:09.573886    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:35:09.573967 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:09.573951    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:35:09.574010 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:09.573989    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:35:10.579935 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:10.579898    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" event={"ID":"bc8e8c4b-b74d-4af5-a2f9-023639b14d42","Type":"ContainerStarted","Data":"f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103"}
Jul 10 03:35:11.010112 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:11.010078    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:35:11.010420 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:11.010387    2576 patch_prober.go:28] interesting pod/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 03:35:11.010472 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:11.010448    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 03:35:12.010350 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:12.010315    2576 patch_prober.go:28] interesting pod/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 03:35:12.010715 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:12.010385    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 03:35:13.010980 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:13.010939    2576 patch_prober.go:28] interesting pod/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 03:35:13.011482 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:13.011013    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 03:35:14.010482 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:14.010444    2576 patch_prober.go:28] interesting pod/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 03:35:14.010649 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:14.010518    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 03:35:15.015428 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:15.015402    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:35:15.015809 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:15.015681    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:35:15.016447 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:15.016425    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:35:15.605349 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:15.605316    2576 generic.go:356] "Generic (PLEG): container finished" podID="58521a5e-8269-4750-bcd4-449df3327cb6" containerID="2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9" exitCode=0
Jul 10 03:35:15.605531 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:15.605398    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" event={"ID":"58521a5e-8269-4750-bcd4-449df3327cb6","Type":"ContainerDied","Data":"2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9"}
Jul 10 03:35:15.605868 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:15.605855    2576 scope.go:117] "RemoveContainer" containerID="2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9"
Jul 10 03:35:15.606898 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:15.606874    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:35:15.606967 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:15.606940    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:35:15.607007 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:15.606975    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 10 03:35:16.612185 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:16.612146    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" event={"ID":"58521a5e-8269-4750-bcd4-449df3327cb6","Type":"ContainerStarted","Data":"cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804"}
Jul 10 03:35:16.977213 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:16.977181    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:16.978471 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:16.978437    2576 patch_prober.go:28] interesting pod/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 03:35:16.978603 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:16.978511    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 03:35:17.976948 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:17.976566    2576 patch_prober.go:28] interesting pod/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 03:35:17.976948 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:17.976622    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 03:35:18.540513 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:18.540474    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:18.976919 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:18.976888    2576 patch_prober.go:28] interesting pod/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 03:35:18.977092 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:18.976942    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 03:35:19.976871 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:19.976837    2576 patch_prober.go:28] interesting pod/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 03:35:19.977068 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:19.976899    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 03:35:20.981254 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:20.981225    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:35:20.982326 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:35:20.982306    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:38:45.257289 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.257208    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"]
Jul 10 03:38:45.257775 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.257677    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="04138795-7174-48b1-971a-5419446a1644" containerName="istio-proxy"
Jul 10 03:38:45.260710 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.260693    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.262961 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.262941    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a7b338\"/\"default-dockercfg-hfx9j\""
Jul 10 03:38:45.264038 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.264016    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a7b338\"/\"kube-root-ca.crt\""
Jul 10 03:38:45.264038 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.264024    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a7b338\"/\"openshift-service-ca.crt\""
Jul 10 03:38:45.264184 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.264049    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a7b338\"/\"sub-test-model-a7b338-kserve-self-signed-certs\""
Jul 10 03:38:45.268868 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.268850    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"]
Jul 10 03:38:45.318983 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.318943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-home\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.319179 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.319066    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-model-cache\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.319179 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.319108    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwtr4\" (UniqueName: \"kubernetes.io/projected/b4107fd9-7fc6-44af-bf09-199b7a702486-kube-api-access-bwtr4\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.319179 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.319140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-dshm\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.319353 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.319316    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-tmp-dir\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.319402 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.319353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4107fd9-7fc6-44af-bf09-199b7a702486-tls-certs\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.420283 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.420245    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-tmp-dir\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.420283 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.420284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4107fd9-7fc6-44af-bf09-199b7a702486-tls-certs\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.420533 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.420311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-home\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.420533 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.420349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-model-cache\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.420533 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.420372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwtr4\" (UniqueName: \"kubernetes.io/projected/b4107fd9-7fc6-44af-bf09-199b7a702486-kube-api-access-bwtr4\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.420533 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.420396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-dshm\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.420817 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.420791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-tmp-dir\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.420882 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.420801    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-home\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.420977 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.420960    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-model-cache\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.422735 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.422709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-dshm\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.422944 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.422927    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4107fd9-7fc6-44af-bf09-199b7a702486-tls-certs\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.427903 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.427884    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwtr4\" (UniqueName: \"kubernetes.io/projected/b4107fd9-7fc6-44af-bf09-199b7a702486-kube-api-access-bwtr4\") pod \"sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") " pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.571368 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.571273    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:45.712388 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:45.712358    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"]
Jul 10 03:38:45.714964 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:38:45.714933    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4107fd9_7fc6_44af_bf09_199b7a702486.slice/crio-691c63f1a174302aad345f315ae4485d147189006be26315a6d76f5afbfb6cb9 WatchSource:0}: Error finding container 691c63f1a174302aad345f315ae4485d147189006be26315a6d76f5afbfb6cb9: Status 404 returned error can't find the container with id 691c63f1a174302aad345f315ae4485d147189006be26315a6d76f5afbfb6cb9
Jul 10 03:38:46.711122 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:46.711081    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl" event={"ID":"b4107fd9-7fc6-44af-bf09-199b7a702486","Type":"ContainerStarted","Data":"d31ab97af5b2547cb697cda0eb8bd5242984dd5e50d678e1f96895925b3aedc9"}
Jul 10 03:38:46.711513 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:46.711135    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl" event={"ID":"b4107fd9-7fc6-44af-bf09-199b7a702486","Type":"ContainerStarted","Data":"691c63f1a174302aad345f315ae4485d147189006be26315a6d76f5afbfb6cb9"}
Jul 10 03:38:46.736548 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:46.736495    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl" podStartSLOduration=1.736479062 podStartE2EDuration="1.736479062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:38:46.735773786 +0000 UTC m=+3536.545039079" watchObservedRunningTime="2026-07-10 03:38:46.736479062 +0000 UTC m=+3536.545744364"
Jul 10 03:38:55.572148 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:55.572112    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:55.572651 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:55.572166    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:55.584676 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:55.584648    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:38:55.770854 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:38:55.770824    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:39:50.962112 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:39:50.962082    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:39:50.968804 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:39:50.968773    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:39:58.350089 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:39:58.350056    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/NetworkManager-dispatcher.service\": RecentStats: unable to find data in memory cache]"
Jul 10 03:40:43.156276 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:43.156244    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"]
Jul 10 03:40:58.176982 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.176923    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl" podUID="b4107fd9-7fc6-44af-bf09-199b7a702486" containerName="main" containerID="cri-o://d31ab97af5b2547cb697cda0eb8bd5242984dd5e50d678e1f96895925b3aedc9" gracePeriod=45
Jul 10 03:40:58.408669 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.408638    2576 generic.go:356] "Generic (PLEG): container finished" podID="b4107fd9-7fc6-44af-bf09-199b7a702486" containerID="d31ab97af5b2547cb697cda0eb8bd5242984dd5e50d678e1f96895925b3aedc9" exitCode=0
Jul 10 03:40:58.408879 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.408709    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl" event={"ID":"b4107fd9-7fc6-44af-bf09-199b7a702486","Type":"ContainerDied","Data":"d31ab97af5b2547cb697cda0eb8bd5242984dd5e50d678e1f96895925b3aedc9"}
Jul 10 03:40:58.435789 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.435723    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:40:58.542710 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.542676    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-home\") pod \"b4107fd9-7fc6-44af-bf09-199b7a702486\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") "
Jul 10 03:40:58.542903 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.542740    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-model-cache\") pod \"b4107fd9-7fc6-44af-bf09-199b7a702486\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") "
Jul 10 03:40:58.542903 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.542835    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4107fd9-7fc6-44af-bf09-199b7a702486-tls-certs\") pod \"b4107fd9-7fc6-44af-bf09-199b7a702486\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") "
Jul 10 03:40:58.542903 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.542865    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwtr4\" (UniqueName: \"kubernetes.io/projected/b4107fd9-7fc6-44af-bf09-199b7a702486-kube-api-access-bwtr4\") pod \"b4107fd9-7fc6-44af-bf09-199b7a702486\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") "
Jul 10 03:40:58.543097 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.542912    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-tmp-dir\") pod \"b4107fd9-7fc6-44af-bf09-199b7a702486\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") "
Jul 10 03:40:58.543097 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.542971    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-dshm\") pod \"b4107fd9-7fc6-44af-bf09-199b7a702486\" (UID: \"b4107fd9-7fc6-44af-bf09-199b7a702486\") "
Jul 10 03:40:58.543097 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.542981    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-home" (OuterVolumeSpecName: "home") pod "b4107fd9-7fc6-44af-bf09-199b7a702486" (UID: "b4107fd9-7fc6-44af-bf09-199b7a702486"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:40:58.543345 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.543033    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-model-cache" (OuterVolumeSpecName: "model-cache") pod "b4107fd9-7fc6-44af-bf09-199b7a702486" (UID: "b4107fd9-7fc6-44af-bf09-199b7a702486"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:40:58.543425 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.543330    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-home\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:40:58.543425 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.543262    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b4107fd9-7fc6-44af-bf09-199b7a702486" (UID: "b4107fd9-7fc6-44af-bf09-199b7a702486"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:40:58.545211 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.545185    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4107fd9-7fc6-44af-bf09-199b7a702486-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4107fd9-7fc6-44af-bf09-199b7a702486" (UID: "b4107fd9-7fc6-44af-bf09-199b7a702486"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:40:58.545353 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.545185    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4107fd9-7fc6-44af-bf09-199b7a702486-kube-api-access-bwtr4" (OuterVolumeSpecName: "kube-api-access-bwtr4") pod "b4107fd9-7fc6-44af-bf09-199b7a702486" (UID: "b4107fd9-7fc6-44af-bf09-199b7a702486"). InnerVolumeSpecName "kube-api-access-bwtr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:40:58.545353 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.545252    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-dshm" (OuterVolumeSpecName: "dshm") pod "b4107fd9-7fc6-44af-bf09-199b7a702486" (UID: "b4107fd9-7fc6-44af-bf09-199b7a702486"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:40:58.644196 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.644158    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-dshm\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:40:58.644196 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.644189    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-model-cache\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:40:58.644196 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.644201    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4107fd9-7fc6-44af-bf09-199b7a702486-tls-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:40:58.644432 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.644209    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwtr4\" (UniqueName: \"kubernetes.io/projected/b4107fd9-7fc6-44af-bf09-199b7a702486-kube-api-access-bwtr4\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:40:58.644432 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:58.644218    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4107fd9-7fc6-44af-bf09-199b7a702486-tmp-dir\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:40:59.414666 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:59.414626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl" event={"ID":"b4107fd9-7fc6-44af-bf09-199b7a702486","Type":"ContainerDied","Data":"691c63f1a174302aad345f315ae4485d147189006be26315a6d76f5afbfb6cb9"}
Jul 10 03:40:59.415092 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:59.414688    2576 scope.go:117] "RemoveContainer" containerID="d31ab97af5b2547cb697cda0eb8bd5242984dd5e50d678e1f96895925b3aedc9"
Jul 10 03:40:59.415092 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:59.414645    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"
Jul 10 03:40:59.430492 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:59.430466    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"]
Jul 10 03:40:59.433940 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:40:59.433916    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a7b338/sub-test-model-a7b338-kserve-784d7c6bd7-l7gtl"]
Jul 10 03:41:00.799896 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:41:00.799858    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4107fd9-7fc6-44af-bf09-199b7a702486" path="/var/lib/kubelet/pods/b4107fd9-7fc6-44af-bf09-199b7a702486/volumes"
Jul 10 03:44:18.602335 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:18.602261    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"]
Jul 10 03:44:18.602838 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:18.602502    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy" containerID="cri-o://f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103" gracePeriod=30
Jul 10 03:44:23.866961 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.866899    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:44:23.980662 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.980622    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-data\") pod \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") "
Jul 10 03:44:23.980662 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.980663    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-token\") pod \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") "
Jul 10 03:44:23.980937 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.980695    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-socket\") pod \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") "
Jul 10 03:44:23.980937 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.980728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-certs\") pod \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") "
Jul 10 03:44:23.980937 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.980791    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwgf5\" (UniqueName: \"kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-kube-api-access-kwgf5\") pod \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") "
Jul 10 03:44:23.980937 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.980855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istiod-ca-cert\") pod \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") "
Jul 10 03:44:23.980937 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.980882    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-envoy\") pod \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") "
Jul 10 03:44:23.980937 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.980910    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-podinfo\") pod \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") "
Jul 10 03:44:23.981234 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.980952    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-credential-socket\") pod \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\" (UID: \"bc8e8c4b-b74d-4af5-a2f9-023639b14d42\") "
Jul 10 03:44:23.981234 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.981087    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bc8e8c4b-b74d-4af5-a2f9-023639b14d42" (UID: "bc8e8c4b-b74d-4af5-a2f9-023639b14d42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:44:23.981413 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.981225    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bc8e8c4b-b74d-4af5-a2f9-023639b14d42" (UID: "bc8e8c4b-b74d-4af5-a2f9-023639b14d42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:44:23.981413 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.981341    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bc8e8c4b-b74d-4af5-a2f9-023639b14d42" (UID: "bc8e8c4b-b74d-4af5-a2f9-023639b14d42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:44:23.981514 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.981483    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:44:23.981514 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.981501    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:44:23.981614 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.981502    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-data" (OuterVolumeSpecName: "istio-data") pod "bc8e8c4b-b74d-4af5-a2f9-023639b14d42" (UID: "bc8e8c4b-b74d-4af5-a2f9-023639b14d42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:44:23.981614 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.981517    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:44:23.981688 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.981641    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bc8e8c4b-b74d-4af5-a2f9-023639b14d42" (UID: "bc8e8c4b-b74d-4af5-a2f9-023639b14d42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:44:23.983122 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.983099    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bc8e8c4b-b74d-4af5-a2f9-023639b14d42" (UID: "bc8e8c4b-b74d-4af5-a2f9-023639b14d42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:44:23.983122 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.983103    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-token" (OuterVolumeSpecName: "istio-token") pod "bc8e8c4b-b74d-4af5-a2f9-023639b14d42" (UID: "bc8e8c4b-b74d-4af5-a2f9-023639b14d42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:44:23.983450 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.983424    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-kube-api-access-kwgf5" (OuterVolumeSpecName: "kube-api-access-kwgf5") pod "bc8e8c4b-b74d-4af5-a2f9-023639b14d42" (UID: "bc8e8c4b-b74d-4af5-a2f9-023639b14d42"). InnerVolumeSpecName "kube-api-access-kwgf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:44:23.983521 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:23.983465    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bc8e8c4b-b74d-4af5-a2f9-023639b14d42" (UID: "bc8e8c4b-b74d-4af5-a2f9-023639b14d42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:44:24.082326 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.082289    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:44:24.082326 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.082318    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:44:24.082326 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.082328    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:44:24.082557 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.082338    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwgf5\" (UniqueName: \"kubernetes.io/projected/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-kube-api-access-kwgf5\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:44:24.082557 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.082347    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:44:24.082557 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.082355    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc8e8c4b-b74d-4af5-a2f9-023639b14d42-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:44:24.490702 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.490663    2576 generic.go:356] "Generic (PLEG): container finished" podID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerID="f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103" exitCode=0
Jul 10 03:44:24.491022 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.490711    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"
Jul 10 03:44:24.491022 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.490758    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" event={"ID":"bc8e8c4b-b74d-4af5-a2f9-023639b14d42","Type":"ContainerDied","Data":"f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103"}
Jul 10 03:44:24.491022 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.490818    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2" event={"ID":"bc8e8c4b-b74d-4af5-a2f9-023639b14d42","Type":"ContainerDied","Data":"1c5ac8b44516dc1dbb00d6faf0e7e97624f5c7571970d999ea354d1232ed7cd2"}
Jul 10 03:44:24.491022 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.490843    2576 scope.go:117] "RemoveContainer" containerID="f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103"
Jul 10 03:44:24.501155 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.501129    2576 scope.go:117] "RemoveContainer" containerID="b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62"
Jul 10 03:44:24.512894 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.512857    2576 scope.go:117] "RemoveContainer" containerID="f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103"
Jul 10 03:44:24.513315 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:44:24.513295    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103\": container with ID starting with f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103 not found: ID does not exist" containerID="f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103"
Jul 10 03:44:24.513374 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.513324    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103"} err="failed to get container status \"f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103\": rpc error: code = NotFound desc = could not find container \"f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103\": container with ID starting with f150a1f0785a5ccc50a1af86209a63e2c3dbd6f45d17e2cfe72f3569672d0103 not found: ID does not exist"
Jul 10 03:44:24.513374 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.513346    2576 scope.go:117] "RemoveContainer" containerID="b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62"
Jul 10 03:44:24.513615 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:44:24.513591    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62\": container with ID starting with b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62 not found: ID does not exist" containerID="b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62"
Jul 10 03:44:24.513669 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.513623    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62"} err="failed to get container status \"b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62\": rpc error: code = NotFound desc = could not find container \"b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62\": container with ID starting with b93f75a8fad677094fd531609444b5242eff236230e7b165948c83880e7edf62 not found: ID does not exist"
Jul 10 03:44:24.516406 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.516382    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"]
Jul 10 03:44:24.518373 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.518354    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a7b338-openshift-default-f8774f6dc-hkdp2"]
Jul 10 03:44:24.799249 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:24.799170    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" path="/var/lib/kubelet/pods/bc8e8c4b-b74d-4af5-a2f9-023639b14d42/volumes"
Jul 10 03:44:50.996145 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:50.996115    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:44:51.003505 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:44:51.003480    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ln8cg_61046ad7-86b4-4781-9728-e7710e740f72/ovn-acl-logging/0.log"
Jul 10 03:45:00.146102 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.146063    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk"]
Jul 10 03:45:00.146578 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.146487    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy"
Jul 10 03:45:00.146578 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.146499    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy"
Jul 10 03:45:00.146578 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.146517    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4107fd9-7fc6-44af-bf09-199b7a702486" containerName="main"
Jul 10 03:45:00.146578 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.146522    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4107fd9-7fc6-44af-bf09-199b7a702486" containerName="main"
Jul 10 03:45:00.146713 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.146592    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy"
Jul 10 03:45:00.146713 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.146601    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4107fd9-7fc6-44af-bf09-199b7a702486" containerName="main"
Jul 10 03:45:00.146713 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.146609    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy"
Jul 10 03:45:00.150846 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.150824    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727585-xpchr"]
Jul 10 03:45:00.150968 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.150942    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk"
Jul 10 03:45:00.151238 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.151219    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy"
Jul 10 03:45:00.151238 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.151238    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc8e8c4b-b74d-4af5-a2f9-023639b14d42" containerName="istio-proxy"
Jul 10 03:45:00.153556 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.153535    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nhxqz\""
Jul 10 03:45:00.154431 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.154299    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-xpchr"
Jul 10 03:45:00.157995 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.157972    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk"]
Jul 10 03:45:00.160790 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.160766    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727585-xpchr"]
Jul 10 03:45:00.165739 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.165718    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk"
Jul 10 03:45:00.172677 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.172659    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-xpchr"
Jul 10 03:45:00.317634 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.317604    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk"]
Jul 10 03:45:00.319358 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:45:00.319326    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a66527_3eea_423d_949e_2d39d11a3274.slice/crio-ab7701d44811b289e7a1b739528221ee822c0ae9d27eb51df2e39c67c3f019ae WatchSource:0}: Error finding container ab7701d44811b289e7a1b739528221ee822c0ae9d27eb51df2e39c67c3f019ae: Status 404 returned error can't find the container with id ab7701d44811b289e7a1b739528221ee822c0ae9d27eb51df2e39c67c3f019ae
Jul 10 03:45:00.543179 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.543152    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727585-xpchr"]
Jul 10 03:45:00.545567 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:45:00.545538    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b274df_0f46_4a82_9ac7_c52beed9b7aa.slice/crio-2c1a5e456cad7195ed0a8ad682e2d0564e00276817bff060f88417765b1e5c9d WatchSource:0}: Error finding container 2c1a5e456cad7195ed0a8ad682e2d0564e00276817bff060f88417765b1e5c9d: Status 404 returned error can't find the container with id 2c1a5e456cad7195ed0a8ad682e2d0564e00276817bff060f88417765b1e5c9d
Jul 10 03:45:00.687860 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.687819    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk" event={"ID":"96a66527-3eea-423d-949e-2d39d11a3274","Type":"ContainerStarted","Data":"3fa419a8b1bc41ab1fa84e0a83b838b71d07685b3640286acae751afe5e7f3f2"}
Jul 10 03:45:00.688046 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.687877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk" event={"ID":"96a66527-3eea-423d-949e-2d39d11a3274","Type":"ContainerStarted","Data":"ab7701d44811b289e7a1b739528221ee822c0ae9d27eb51df2e39c67c3f019ae"}
Jul 10 03:45:00.689106 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.689082    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-xpchr" event={"ID":"a6b274df-0f46-4a82-9ac7-c52beed9b7aa","Type":"ContainerStarted","Data":"2c1a5e456cad7195ed0a8ad682e2d0564e00276817bff060f88417765b1e5c9d"}
Jul 10 03:45:00.705150 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:00.705102    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk" podStartSLOduration=0.705088227 podStartE2EDuration="705.088227ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:45:00.703078126 +0000 UTC m=+3910.512343429" watchObservedRunningTime="2026-07-10 03:45:00.705088227 +0000 UTC m=+3910.514353530"
Jul 10 03:45:01.695511 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:01.695472    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-xpchr" event={"ID":"a6b274df-0f46-4a82-9ac7-c52beed9b7aa","Type":"ContainerStarted","Data":"b39dbe46a6bf14e02fa00057358698f8e679b5ce4ef451035f083783b563ed4f"}
Jul 10 03:45:01.711910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:01.711860    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-xpchr" podStartSLOduration=1.711843695 podStartE2EDuration="1.711843695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:45:01.709938394 +0000 UTC m=+3911.519203695" watchObservedRunningTime="2026-07-10 03:45:01.711843695 +0000 UTC m=+3911.521108996"
Jul 10 03:45:24.857559 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:24.857519    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk"]
Jul 10 03:45:24.857964 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:24.857722    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk" podUID="96a66527-3eea-423d-949e-2d39d11a3274" containerName="cleanup" containerID="cri-o://3fa419a8b1bc41ab1fa84e0a83b838b71d07685b3640286acae751afe5e7f3f2" gracePeriod=30
Jul 10 03:45:31.305416 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:31.305372    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"]
Jul 10 03:45:31.305921 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:31.305627    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy" containerID="cri-o://cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804" gracePeriod=30
Jul 10 03:45:35.984419 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:35.984378    2576 patch_prober.go:28] interesting pod/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 03:45:35.984899 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:35.984462    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 03:45:36.562601 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.562576    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:45:36.702296 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702206    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6lc6\" (UniqueName: \"kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-kube-api-access-k6lc6\") pod \"58521a5e-8269-4750-bcd4-449df3327cb6\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") "
Jul 10 03:45:36.702296 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702261    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-data\") pod \"58521a5e-8269-4750-bcd4-449df3327cb6\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") "
Jul 10 03:45:36.702296 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702291    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-istio-token\") pod \"58521a5e-8269-4750-bcd4-449df3327cb6\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") "
Jul 10 03:45:36.702583 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702307    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-credential-socket\") pod \"58521a5e-8269-4750-bcd4-449df3327cb6\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") "
Jul 10 03:45:36.702583 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702358    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58521a5e-8269-4750-bcd4-449df3327cb6-istio-podinfo\") pod \"58521a5e-8269-4750-bcd4-449df3327cb6\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") "
Jul 10 03:45:36.702583 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702534    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-certs\") pod \"58521a5e-8269-4750-bcd4-449df3327cb6\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") "
Jul 10 03:45:36.702777 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702613    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-envoy\") pod \"58521a5e-8269-4750-bcd4-449df3327cb6\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") "
Jul 10 03:45:36.702777 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702672    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-socket\") pod \"58521a5e-8269-4750-bcd4-449df3327cb6\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") "
Jul 10 03:45:36.702777 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702690    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "58521a5e-8269-4750-bcd4-449df3327cb6" (UID: "58521a5e-8269-4750-bcd4-449df3327cb6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:45:36.702777 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702726    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58521a5e-8269-4750-bcd4-449df3327cb6-istiod-ca-cert\") pod \"58521a5e-8269-4750-bcd4-449df3327cb6\" (UID: \"58521a5e-8269-4750-bcd4-449df3327cb6\") "
Jul 10 03:45:36.702999 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702771    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "58521a5e-8269-4750-bcd4-449df3327cb6" (UID: "58521a5e-8269-4750-bcd4-449df3327cb6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:45:36.702999 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.702926    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "58521a5e-8269-4750-bcd4-449df3327cb6" (UID: "58521a5e-8269-4750-bcd4-449df3327cb6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:45:36.703203 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.703180    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-data" (OuterVolumeSpecName: "istio-data") pod "58521a5e-8269-4750-bcd4-449df3327cb6" (UID: "58521a5e-8269-4750-bcd4-449df3327cb6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:45:36.703268 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.703207    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-credential-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:45:36.703268 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.703230    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-certs\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:45:36.703268 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.703237    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58521a5e-8269-4750-bcd4-449df3327cb6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "58521a5e-8269-4750-bcd4-449df3327cb6" (UID: "58521a5e-8269-4750-bcd4-449df3327cb6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:45:36.703268 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.703244    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-workload-socket\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:45:36.704541 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.704513    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-kube-api-access-k6lc6" (OuterVolumeSpecName: "kube-api-access-k6lc6") pod "58521a5e-8269-4750-bcd4-449df3327cb6" (UID: "58521a5e-8269-4750-bcd4-449df3327cb6"). InnerVolumeSpecName "kube-api-access-k6lc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:45:36.704716 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.704696    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "58521a5e-8269-4750-bcd4-449df3327cb6" (UID: "58521a5e-8269-4750-bcd4-449df3327cb6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:45:36.704954 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.704936    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58521a5e-8269-4750-bcd4-449df3327cb6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "58521a5e-8269-4750-bcd4-449df3327cb6" (UID: "58521a5e-8269-4750-bcd4-449df3327cb6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 03:45:36.704954 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.704944    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-istio-token" (OuterVolumeSpecName: "istio-token") pod "58521a5e-8269-4750-bcd4-449df3327cb6" (UID: "58521a5e-8269-4750-bcd4-449df3327cb6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:45:36.804125 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.804093    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-data\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:45:36.804125 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.804118    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-istio-token\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:45:36.804125 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.804128    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58521a5e-8269-4750-bcd4-449df3327cb6-istio-podinfo\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:45:36.804351 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.804136    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58521a5e-8269-4750-bcd4-449df3327cb6-istio-envoy\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:45:36.804351 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.804144    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58521a5e-8269-4750-bcd4-449df3327cb6-istiod-ca-cert\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:45:36.804351 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.804152    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6lc6\" (UniqueName: \"kubernetes.io/projected/58521a5e-8269-4750-bcd4-449df3327cb6-kube-api-access-k6lc6\") on node \"ip-10-0-143-93.ec2.internal\" DevicePath \"\""
Jul 10 03:45:36.878824 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.878792    2576 generic.go:356] "Generic (PLEG): container finished" podID="58521a5e-8269-4750-bcd4-449df3327cb6" containerID="cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804" exitCode=0
Jul 10 03:45:36.879008 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.878852    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"
Jul 10 03:45:36.879008 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.878879    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" event={"ID":"58521a5e-8269-4750-bcd4-449df3327cb6","Type":"ContainerDied","Data":"cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804"}
Jul 10 03:45:36.879008 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.878930    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25" event={"ID":"58521a5e-8269-4750-bcd4-449df3327cb6","Type":"ContainerDied","Data":"42538e86257b85196c6623fa91fea90896a61d375b89c847991f16286887fbca"}
Jul 10 03:45:36.879008 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.878952    2576 scope.go:117] "RemoveContainer" containerID="cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804"
Jul 10 03:45:36.888528 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.888512    2576 scope.go:117] "RemoveContainer" containerID="2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9"
Jul 10 03:45:36.896071 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.896044    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"]
Jul 10 03:45:36.896938 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.896920    2576 scope.go:117] "RemoveContainer" containerID="cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804"
Jul 10 03:45:36.897288 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:45:36.897235    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804\": container with ID starting with cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804 not found: ID does not exist" containerID="cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804"
Jul 10 03:45:36.897427 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.897299    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804"} err="failed to get container status \"cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804\": rpc error: code = NotFound desc = could not find container \"cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804\": container with ID starting with cc2a356fcd9c3c1f90127456bd9b7219055cf88b34bba127708c114eb60fb804 not found: ID does not exist"
Jul 10 03:45:36.897427 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.897354    2576 scope.go:117] "RemoveContainer" containerID="2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9"
Jul 10 03:45:36.897894 ip-10-0-143-93 kubenswrapper[2576]: E0710 03:45:36.897651    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9\": container with ID starting with 2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9 not found: ID does not exist" containerID="2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9"
Jul 10 03:45:36.897894 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.897707    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9"} err="failed to get container status \"2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9\": rpc error: code = NotFound desc = could not find container \"2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9\": container with ID starting with 2a5df13b4f9bf5fbd826932f92b8c9e20cb4a2da00ed2c9da734ad5fc573fcc9 not found: ID does not exist"
Jul 10 03:45:36.899907 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:36.899888    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-17c732-openshift-default-68cb85f679-szm25"]
Jul 10 03:45:37.180189 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:37.180157    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-g4q8p_527b17dc-ebab-46c3-9987-919b9b63f0b9/manager/0.log"
Jul 10 03:45:38.084092 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:38.084061    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68647b69dd-r2qwd_e9962e03-c473-4429-a9f7-36a28710d2d4/manager/0.log"
Jul 10 03:45:38.090836 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:38.090810    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68647b69dd-r2qwd_e9962e03-c473-4429-a9f7-36a28710d2d4/copy-manifests/0.log"
Jul 10 03:45:38.569054 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:38.569027    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29727585-xpchr_a6b274df-0f46-4a82-9ac7-c52beed9b7aa/cleanup/0.log"
Jul 10 03:45:38.691070 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:38.691040    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk_96a66527-3eea-423d-949e-2d39d11a3274/cleanup/0.log"
Jul 10 03:45:38.800407 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:38.800371    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" path="/var/lib/kubelet/pods/58521a5e-8269-4750-bcd4-449df3327cb6/volumes"
Jul 10 03:45:38.809387 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:38.809359    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-sjrx2_7229b95d-6f40-47e6-bfbc-a13edde8d16c/postgres/0.log"
Jul 10 03:45:40.318672 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:40.318642    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-p9l66_6c165bf8-331d-4fa9-a869-f5a4caaa5949/manager/0.log"
Jul 10 03:45:40.679181 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:40.679106    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-cw67h_4f80dac4-aff7-4544-8056-1479d748a33d/registry-server/0.log"
Jul 10 03:45:41.056504 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:41.056468    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-h2jwk_e7dd2af0-db13-4782-88d3-4afcadd2202b/manager/0.log"
Jul 10 03:45:41.442090 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:41.442003    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dv6pn_6d9b0605-5ed4-4f3f-ad3e-748978c988e1/istio-proxy/0.log"
Jul 10 03:45:41.953466 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:41.953435    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j9rbd_f12035fc-fa7a-4e5b-bb17-870f656e5c2e/istio-proxy/1.log"
Jul 10 03:45:42.338043 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:42.338012    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76c954fdf4-rc6fn_b0ecac30-e48a-4824-b238-ab3453895b35/router/0.log"
Jul 10 03:45:43.090552 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:43.090521    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtvltr_78b7dd38-5b46-4eda-9aa8-660669952819/main/0.log"
Jul 10 03:45:50.879699 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:50.879666    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-npnft_3d813814-8b23-41d4-852b-ff842e8a5771/global-pull-secret-syncer/0.log"
Jul 10 03:45:51.004066 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:51.004041    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gphcf_a0179f78-2a2a-4f3f-bbd3-bcb655dd6e10/konnectivity-agent/0.log"
Jul 10 03:45:51.136785 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:51.136671    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-93.ec2.internal_7267bfa133f7ae1ca9f615a6770bb5df/haproxy/0.log"
Jul 10 03:45:54.972685 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:54.972645    2576 generic.go:356] "Generic (PLEG): container finished" podID="96a66527-3eea-423d-949e-2d39d11a3274" containerID="3fa419a8b1bc41ab1fa84e0a83b838b71d07685b3640286acae751afe5e7f3f2" exitCode=137
Jul 10 03:45:54.973167 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:54.972698    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk" event={"ID":"96a66527-3eea-423d-949e-2d39d11a3274","Type":"ContainerDied","Data":"3fa419a8b1bc41ab1fa84e0a83b838b71d07685b3640286acae751afe5e7f3f2"}
Jul 10 03:45:55.065691 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:55.065665    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk"
Jul 10 03:45:55.979022 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:55.978982    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk"
Jul 10 03:45:55.979022 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:55.979016    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk" event={"ID":"96a66527-3eea-423d-949e-2d39d11a3274","Type":"ContainerDied","Data":"ab7701d44811b289e7a1b739528221ee822c0ae9d27eb51df2e39c67c3f019ae"}
Jul 10 03:45:55.979517 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:55.979083    2576 scope.go:117] "RemoveContainer" containerID="3fa419a8b1bc41ab1fa84e0a83b838b71d07685b3640286acae751afe5e7f3f2"
Jul 10 03:45:56.002734 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:56.002703    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk"]
Jul 10 03:45:56.004657 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:56.004630    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-17c732-29727585-4gjwk"]
Jul 10 03:45:56.561448 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:56.561414    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-p9l66_6c165bf8-331d-4fa9-a869-f5a4caaa5949/manager/0.log"
Jul 10 03:45:56.676042 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:56.676009    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-cw67h_4f80dac4-aff7-4544-8056-1479d748a33d/registry-server/0.log"
Jul 10 03:45:56.800187 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:56.800156    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96a66527-3eea-423d-949e-2d39d11a3274" path="/var/lib/kubelet/pods/96a66527-3eea-423d-949e-2d39d11a3274/volumes"
Jul 10 03:45:56.899501 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:56.899409    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-h2jwk_e7dd2af0-db13-4782-88d3-4afcadd2202b/manager/0.log"
Jul 10 03:45:58.630841 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.630805    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_de855ae6-469b-412b-9658-aa3731d39890/alertmanager/0.log"
Jul 10 03:45:58.654933 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.654907    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_de855ae6-469b-412b-9658-aa3731d39890/config-reloader/0.log"
Jul 10 03:45:58.678583 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.678518    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_de855ae6-469b-412b-9658-aa3731d39890/kube-rbac-proxy-web/0.log"
Jul 10 03:45:58.703686 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.703670    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_de855ae6-469b-412b-9658-aa3731d39890/kube-rbac-proxy/0.log"
Jul 10 03:45:58.726532 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.726512    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_de855ae6-469b-412b-9658-aa3731d39890/kube-rbac-proxy-metric/0.log"
Jul 10 03:45:58.750678 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.750653    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_de855ae6-469b-412b-9658-aa3731d39890/prom-label-proxy/0.log"
Jul 10 03:45:58.774512 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.774491    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_de855ae6-469b-412b-9658-aa3731d39890/init-config-reloader/0.log"
Jul 10 03:45:58.819468 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.819445    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-4rvpk_8be658e5-7cb3-4096-b810-3b2791998605/cluster-monitoring-operator/0.log"
Jul 10 03:45:58.845663 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.845644    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-g2zrn_937ec168-89ef-4178-aeb2-c2e1c6c910b2/kube-state-metrics/0.log"
Jul 10 03:45:58.869463 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.869442    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-g2zrn_937ec168-89ef-4178-aeb2-c2e1c6c910b2/kube-rbac-proxy-main/0.log"
Jul 10 03:45:58.894868 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:58.894849    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-g2zrn_937ec168-89ef-4178-aeb2-c2e1c6c910b2/kube-rbac-proxy-self/0.log"
Jul 10 03:45:59.075464 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.075435    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k4hqs_d8f513ce-213d-41e2-8d62-c6b811c0afce/node-exporter/0.log"
Jul 10 03:45:59.098175 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.098146    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k4hqs_d8f513ce-213d-41e2-8d62-c6b811c0afce/kube-rbac-proxy/0.log"
Jul 10 03:45:59.123650 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.123626    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k4hqs_d8f513ce-213d-41e2-8d62-c6b811c0afce/init-textfile/0.log"
Jul 10 03:45:59.330162 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.330073    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_adabbf19-0686-4bf0-9d96-732f673a5fa1/prometheus/0.log"
Jul 10 03:45:59.354555 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.354531    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_adabbf19-0686-4bf0-9d96-732f673a5fa1/config-reloader/0.log"
Jul 10 03:45:59.381598 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.381575    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_adabbf19-0686-4bf0-9d96-732f673a5fa1/thanos-sidecar/0.log"
Jul 10 03:45:59.404766 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.404727    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_adabbf19-0686-4bf0-9d96-732f673a5fa1/kube-rbac-proxy-web/0.log"
Jul 10 03:45:59.430702 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.430666    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_adabbf19-0686-4bf0-9d96-732f673a5fa1/kube-rbac-proxy/0.log"
Jul 10 03:45:59.455915 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.455888    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_adabbf19-0686-4bf0-9d96-732f673a5fa1/kube-rbac-proxy-thanos/0.log"
Jul 10 03:45:59.479492 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.479462    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_adabbf19-0686-4bf0-9d96-732f673a5fa1/init-config-reloader/0.log"
Jul 10 03:45:59.508150 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.508117    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-g2w74_0e7a9540-db48-4cec-9934-b49772d03eed/prometheus-operator/0.log"
Jul 10 03:45:59.533535 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.533514    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-g2w74_0e7a9540-db48-4cec-9934-b49772d03eed/kube-rbac-proxy/0.log"
Jul 10 03:45:59.567661 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.567635    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-8f5zn_a4f5153b-b5f2-4eb5-a79b-a99dcd3e6d26/prometheus-operator-admission-webhook/0.log"
Jul 10 03:45:59.686380 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.686306    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657c76b9bf-mw97d_8770b5b7-4639-4784-926c-f4fccb16d00c/thanos-query/0.log"
Jul 10 03:45:59.709290 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.709263    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657c76b9bf-mw97d_8770b5b7-4639-4784-926c-f4fccb16d00c/kube-rbac-proxy-web/0.log"
Jul 10 03:45:59.732905 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.732881    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657c76b9bf-mw97d_8770b5b7-4639-4784-926c-f4fccb16d00c/kube-rbac-proxy/0.log"
Jul 10 03:45:59.755637 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.755607    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657c76b9bf-mw97d_8770b5b7-4639-4784-926c-f4fccb16d00c/prom-label-proxy/0.log"
Jul 10 03:45:59.778244 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.778224    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657c76b9bf-mw97d_8770b5b7-4639-4784-926c-f4fccb16d00c/kube-rbac-proxy-rules/0.log"
Jul 10 03:45:59.800338 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:45:59.800318    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657c76b9bf-mw97d_8770b5b7-4639-4784-926c-f4fccb16d00c/kube-rbac-proxy-metrics/0.log"
Jul 10 03:46:00.011067 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:00.011036    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727585-xpchr"]
Jul 10 03:46:00.011246 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:00.011221    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727585-xpchr" podUID="a6b274df-0f46-4a82-9ac7-c52beed9b7aa" containerName="cleanup" containerID="cri-o://b39dbe46a6bf14e02fa00057358698f8e679b5ce4ef451035f083783b563ed4f" gracePeriod=30
Jul 10 03:46:00.980640 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:00.980610    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-ktzsj_3d2951ef-9d07-4f7a-8d18-6e2a0144e216/networking-console-plugin/0.log"
Jul 10 03:46:01.388298 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.388214    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"]
Jul 10 03:46:01.388682 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.388668    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy"
Jul 10 03:46:01.388727 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.388684    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy"
Jul 10 03:46:01.388727 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.388697    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96a66527-3eea-423d-949e-2d39d11a3274" containerName="cleanup"
Jul 10 03:46:01.388727 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.388703    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="96a66527-3eea-423d-949e-2d39d11a3274" containerName="cleanup"
Jul 10 03:46:01.388727 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.388714    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy"
Jul 10 03:46:01.388727 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.388719    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy"
Jul 10 03:46:01.388949 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.388793    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy"
Jul 10 03:46:01.388949 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.388801    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="58521a5e-8269-4750-bcd4-449df3327cb6" containerName="istio-proxy"
Jul 10 03:46:01.388949 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.388810    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="96a66527-3eea-423d-949e-2d39d11a3274" containerName="cleanup"
Jul 10 03:46:01.393637 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.393612    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.397463 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.397412    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4zdlx\"/\"default-dockercfg-6bsvz\""
Jul 10 03:46:01.397463 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.397415    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4zdlx\"/\"openshift-service-ca.crt\""
Jul 10 03:46:01.397463 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.397458    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4zdlx\"/\"kube-root-ca.crt\""
Jul 10 03:46:01.398846 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.398824    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"]
Jul 10 03:46:01.432267 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.432237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-sys\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.432448 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.432284    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-lib-modules\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.432448 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.432383    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-podres\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.432448 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.432444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4srf\" (UniqueName: \"kubernetes.io/projected/f389da52-521a-4562-9730-82052081d07b-kube-api-access-w4srf\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.432564 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.432467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-proc\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.533542 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.533505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-podres\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.533793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.533567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4srf\" (UniqueName: \"kubernetes.io/projected/f389da52-521a-4562-9730-82052081d07b-kube-api-access-w4srf\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.533793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.533587    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-proc\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.533793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.533611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-sys\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.533793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.533643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-lib-modules\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.533793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.533712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-podres\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.533793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.533736    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-sys\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.533793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.533712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-proc\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.533793 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.533798    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f389da52-521a-4562-9730-82052081d07b-lib-modules\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.541601 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.541578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4srf\" (UniqueName: \"kubernetes.io/projected/f389da52-521a-4562-9730-82052081d07b-kube-api-access-w4srf\") pod \"perf-node-gather-daemonset-kxdpc\" (UID: \"f389da52-521a-4562-9730-82052081d07b\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.705585 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.705496    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:01.841612 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:01.841583    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"]
Jul 10 03:46:01.843918 ip-10-0-143-93 kubenswrapper[2576]: W0710 03:46:01.843887    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf389da52_521a_4562_9730_82052081d07b.slice/crio-3156c18478b7f3f477b800117c16b2fcb053787e0e9753f0afd3e86474f1196c WatchSource:0}: Error finding container 3156c18478b7f3f477b800117c16b2fcb053787e0e9753f0afd3e86474f1196c: Status 404 returned error can't find the container with id 3156c18478b7f3f477b800117c16b2fcb053787e0e9753f0afd3e86474f1196c
Jul 10 03:46:02.014395 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:02.014354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc" event={"ID":"f389da52-521a-4562-9730-82052081d07b","Type":"ContainerStarted","Data":"5fb069f2ab9b9b98bab068bad7336453e4ceeae05de15b284129abe6405c8770"}
Jul 10 03:46:02.014795 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:02.014419    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc" event={"ID":"f389da52-521a-4562-9730-82052081d07b","Type":"ContainerStarted","Data":"3156c18478b7f3f477b800117c16b2fcb053787e0e9753f0afd3e86474f1196c"}
Jul 10 03:46:02.014795 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:02.014530    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:02.031715 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:02.031666    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc" podStartSLOduration=1.031649459 podStartE2EDuration="1.031649459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:46:02.029511925 +0000 UTC m=+3971.838777230" watchObservedRunningTime="2026-07-10 03:46:02.031649459 +0000 UTC m=+3971.840914762"
Jul 10 03:46:02.154287 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:02.154258    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-fdd69469c-lsvf9_da7e1773-e30d-4956-82e7-6128cbcdf29f/console/0.log"
Jul 10 03:46:03.877682 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:03.877653    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-979w2_b15416cc-0665-46f9-af55-f1eb77451bbc/dns/0.log"
Jul 10 03:46:03.902163 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:03.902135    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-979w2_b15416cc-0665-46f9-af55-f1eb77451bbc/kube-rbac-proxy/0.log"
Jul 10 03:46:04.011219 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:04.011191    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cfmzm_22008c3f-7d3c-4151-910c-8950c916a84c/dns-node-resolver/0.log"
Jul 10 03:46:04.735490 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:04.735461    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r7k5d_01980251-990d-4898-b259-8a05b74de2fc/node-ca/0.log"
Jul 10 03:46:05.674416 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:05.674387    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dv6pn_6d9b0605-5ed4-4f3f-ad3e-748978c988e1/istio-proxy/0.log"
Jul 10 03:46:05.877178 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:05.877148    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j9rbd_f12035fc-fa7a-4e5b-bb17-870f656e5c2e/istio-proxy/0.log"
Jul 10 03:46:05.911084 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:05.911057    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j9rbd_f12035fc-fa7a-4e5b-bb17-870f656e5c2e/istio-proxy/1.log"
Jul 10 03:46:05.993556 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:05.993527    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76c954fdf4-rc6fn_b0ecac30-e48a-4824-b238-ab3453895b35/router/0.log"
Jul 10 03:46:06.761708 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:06.761676    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fktxn_b5c7dab0-cc72-4598-8761-2256c99bb318/serve-healthcheck-canary/0.log"
Jul 10 03:46:07.263102 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:07.263066    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-lg5nc_e2aff6a6-f610-4b8f-93d0-c983270ee97a/insights-operator/0.log"
Jul 10 03:46:07.264055 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:07.264035    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-lg5nc_e2aff6a6-f610-4b8f-93d0-c983270ee97a/insights-operator/1.log"
Jul 10 03:46:07.362648 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:07.362615    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cfxrv_239684b3-706b-4ec0-b267-e7f179b0ba03/kube-rbac-proxy/0.log"
Jul 10 03:46:07.385896 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:07.385869    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cfxrv_239684b3-706b-4ec0-b267-e7f179b0ba03/exporter/0.log"
Jul 10 03:46:07.408052 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:07.408028    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cfxrv_239684b3-706b-4ec0-b267-e7f179b0ba03/extractor/0.log"
Jul 10 03:46:08.029199 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:08.029168    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-kxdpc"
Jul 10 03:46:09.610896 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:09.610865    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-g4q8p_527b17dc-ebab-46c3-9987-919b9b63f0b9/manager/0.log"
Jul 10 03:46:10.170464 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:10.170436    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68647b69dd-r2qwd_e9962e03-c473-4429-a9f7-36a28710d2d4/manager/0.log"
Jul 10 03:46:10.193410 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:10.193385    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68647b69dd-r2qwd_e9962e03-c473-4429-a9f7-36a28710d2d4/copy-manifests/0.log"
Jul 10 03:46:11.695278 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:11.695249    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-b767bdcf8-frd25_d15c3296-a7fc-4503-a551-a10ce7fcaa03/manager/0.log"
Jul 10 03:46:12.162036 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:12.162006    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dv6pn_6d9b0605-5ed4-4f3f-ad3e-748978c988e1/istio-proxy/0.log"
Jul 10 03:46:12.318212 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:12.318176    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j9rbd_f12035fc-fa7a-4e5b-bb17-870f656e5c2e/istio-proxy/0.log"
Jul 10 03:46:12.350941 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:12.350910    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j9rbd_f12035fc-fa7a-4e5b-bb17-870f656e5c2e/istio-proxy/1.log"
Jul 10 03:46:12.411149 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:12.411117    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76c954fdf4-rc6fn_b0ecac30-e48a-4824-b238-ab3453895b35/router/0.log"
Jul 10 03:46:13.692568 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:13.692532    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dv6pn_6d9b0605-5ed4-4f3f-ad3e-748978c988e1/istio-proxy/0.log"
Jul 10 03:46:13.854094 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:13.854065    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j9rbd_f12035fc-fa7a-4e5b-bb17-870f656e5c2e/istio-proxy/0.log"
Jul 10 03:46:13.884030 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:13.883995    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j9rbd_f12035fc-fa7a-4e5b-bb17-870f656e5c2e/istio-proxy/1.log"
Jul 10 03:46:13.946269 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:13.946187    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76c954fdf4-rc6fn_b0ecac30-e48a-4824-b238-ab3453895b35/router/0.log"
Jul 10 03:46:18.401910 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:18.401882    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmb92_228cc28c-9b29-4a85-a465-040c721653fa/kube-multus-additional-cni-plugins/0.log"
Jul 10 03:46:18.425563 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:18.425538    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmb92_228cc28c-9b29-4a85-a465-040c721653fa/egress-router-binary-copy/0.log"
Jul 10 03:46:18.448455 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:18.448425    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmb92_228cc28c-9b29-4a85-a465-040c721653fa/cni-plugins/0.log"
Jul 10 03:46:18.470697 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:18.470670    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmb92_228cc28c-9b29-4a85-a465-040c721653fa/bond-cni-plugin/0.log"
Jul 10 03:46:18.493286 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:18.493259    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmb92_228cc28c-9b29-4a85-a465-040c721653fa/routeoverride-cni/0.log"
Jul 10 03:46:18.515821 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:18.515794    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmb92_228cc28c-9b29-4a85-a465-040c721653fa/whereabouts-cni-bincopy/0.log"
Jul 10 03:46:18.538422 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:18.538399    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmb92_228cc28c-9b29-4a85-a465-040c721653fa/whereabouts-cni/0.log"
Jul 10 03:46:18.753318 ip-10-0-143-93 kubenswrapper[2576]: I0710 03:46:18.753281    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-k2s62_09ab858c-346a-4ccd-9834-627796c08d79/kube-multus/0.log"