Jul 02 14:15:34.761458 ip-10-0-138-134 systemd[1]: Starting Kubernetes Kubelet... Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.191308 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195043 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195056 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195061 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195065 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195068 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195072 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195075 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:15:35.381969 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195078 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:15:35.273484 ip-10-0-138-134 systemd[1]: Started Kubernetes Kubelet. Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195081 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195084 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195087 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195089 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195092 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195095 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195098 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195100 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195103 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195106 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195109 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195112 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195119 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195122 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195125 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195128 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195131 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195134 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195137 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:15:35.393392 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195140 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195142 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195164 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195170 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195174 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195178 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195181 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195184 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195186 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195189 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195192 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195195 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195197 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195200 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195202 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195205 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195207 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195210 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195213 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195216 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:15:35.394572 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195220 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195223 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195225 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195228 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195230 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195233 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195236 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195239 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195241 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195244 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195247 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195249 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195252 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195254 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195256 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195259 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195262 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195265 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195267 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195270 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:15:35.395622 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195273 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195275 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195277 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195280 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195282 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195285 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195287 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195290 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195292 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195295 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195298 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195300 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195303 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195305 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195308 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195311 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195313 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195316 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195318 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:15:35.396517 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195321 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195324 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195327 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195330 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195333 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195335 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195339 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195343 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195347 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195350 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195353 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195357 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.195361 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196062 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196069 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196072 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196075 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196078 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196080 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:15:35.397412 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196083 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196086 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196088 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196092 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196095 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196098 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196101 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196104 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196107 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196109 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196112 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196115 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196118 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196121 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196124 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196126 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196128 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196131 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196134 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196136 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:15:35.398513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196139 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196142 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196160 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196163 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196166 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196168 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196171 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196173 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196176 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196178 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196181 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196184 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196186 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196189 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196191 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196194 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196196 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196199 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196202 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196205 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:15:35.399601 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196208 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196211 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196213 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196215 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196218 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196221 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196223 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196226 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196229 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196231 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196233 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196236 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196239 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196241 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196244 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196246 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196249 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196251 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196256 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196260 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:15:35.400671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196263 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196266 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196269 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196271 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196275 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196279 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196283 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196291 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196294 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196297 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196299 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196302 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196305 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196307 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196310 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196313 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196315 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196318 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196320 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196323 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:15:35.401561 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196326 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196329 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196333 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196336 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196339 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196341 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196344 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196346 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196349 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196351 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196354 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196356 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196359 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196734 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196739 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196742 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196745 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196747 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196750 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:15:35.402468 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196754 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196757 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196760 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196763 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196765 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196768 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196771 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196776 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196778 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196781 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196783 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196786 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196788 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196791 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196794 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196797 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196800 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196803 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196807 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:15:35.403588 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196810 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196812 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196815 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196817 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196820 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196822 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196825 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196827 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196830 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196832 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196835 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196838 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196840 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196844 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196847 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196850 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196852 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196855 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196857 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196860 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:15:35.404479 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196863 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196866 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196869 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196871 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196874 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196876 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196879 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196882 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196884 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196887 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196889 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196892 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196895 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196897 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196899 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196903 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196907 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196910 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196913 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:15:35.405380 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196916 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196918 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196920 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196923 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196925 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196928 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196932 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196934 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196937 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196939 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196942 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196945 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196947 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196951 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196953 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196958 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196961 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196964 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196967 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196970 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:15:35.406402 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196973 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196976 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196978 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196981 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196983 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196986 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196988 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196991 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196993 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196996 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.196998 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197001 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197003 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197005 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197008 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197080 2569 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197087 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197093 2569 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197098 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197102 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197106 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 14:15:35.407280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197110 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197114 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197117 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197120 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197125 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197129 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197132 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197135 2569 flags.go:64] FLAG: --cgroup-root="" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197138 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197141 2569 flags.go:64] FLAG: --client-ca-file="" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197157 2569 flags.go:64] FLAG: --cloud-provider="external" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197160 2569 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197164 2569 flags.go:64] FLAG: --cluster-domain="" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197167 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197171 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197174 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197177 2569 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197181 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197184 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197187 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197190 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197194 2569 flags.go:64] FLAG: --contention-profiling="false" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197197 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197201 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 14:15:35.408171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197227 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197231 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197236 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197239 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197242 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197247 2569 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197250 2569 flags.go:64] FLAG: --enable-server="true" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197253 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197257 2569 flags.go:64] FLAG: --event-burst="100" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197261 2569 flags.go:64] FLAG: --event-qps="50" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197264 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197267 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197271 2569 flags.go:64] FLAG: --eviction-hard="" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197275 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197277 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197280 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197283 2569 flags.go:64] FLAG: --eviction-soft="" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197286 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197289 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197292 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197305 2569 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197308 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197312 2569 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197315 2569 flags.go:64] FLAG: --feature-gates="" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197319 2569 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 14:15:35.409487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197322 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197325 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197328 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197331 2569 flags.go:64] FLAG: --healthz-port="10248" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197334 2569 flags.go:64] FLAG: --help="false" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197338 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-138-134.ec2.internal" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197341 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197344 2569 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197347 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197350 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197354 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197357 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197361 2569 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197364 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197366 2569 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197369 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197373 2569 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197375 2569 flags.go:64] FLAG: --kube-reserved="" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197378 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197383 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197386 2569 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197388 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197391 2569 flags.go:64] FLAG: --lock-file="" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197394 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 14:15:35.410602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197397 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197400 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197405 2569 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197409 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197412 2569 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197414 2569 flags.go:64] FLAG: --logging-format="text" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197418 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197421 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197423 2569 flags.go:64] FLAG: --manifest-url="" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197426 2569 flags.go:64] FLAG: --manifest-url-header="" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197430 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197433 2569 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197437 2569 flags.go:64] FLAG: --max-pods="110" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197440 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197443 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197446 2569 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197449 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197452 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197455 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197457 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197465 2569 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197468 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197471 2569 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 14:15:35.411685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197474 2569 flags.go:64] FLAG: --pod-cidr="" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197477 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197483 2569 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197486 2569 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197489 2569 flags.go:64] FLAG: --pods-per-core="0" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197493 2569 flags.go:64] FLAG: --port="10250" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197496 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197499 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0292336e6064b6b86" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197502 2569 flags.go:64] FLAG: --qos-reserved="" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197505 2569 flags.go:64] FLAG: --read-only-port="10255" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197508 2569 flags.go:64] FLAG: --register-node="true" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197511 2569 flags.go:64] FLAG: --register-with-taints="" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197515 2569 flags.go:64] FLAG: --registry-burst="10" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197518 2569 flags.go:64] FLAG: --registry-qps="5" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197521 2569 flags.go:64] FLAG: --reserved-cpus="" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197524 2569 flags.go:64] FLAG: --reserved-memory="" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197531 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197534 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197537 2569 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197540 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197543 2569 flags.go:64] FLAG: --runonce="false" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197547 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197550 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197553 2569 flags.go:64] FLAG: --seccomp-default="false" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197555 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 14:15:35.415159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197559 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197562 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197565 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197568 2569 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197572 2569 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197575 2569 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197577 2569 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197580 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197583 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197586 2569 flags.go:64] FLAG: --system-cgroups="" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197589 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197594 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197598 2569 flags.go:64] FLAG: --tls-cert-file="" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197601 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197605 2569 flags.go:64] FLAG: --tls-min-version="" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197608 2569 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197610 2569 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197613 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197616 2569 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197619 2569 flags.go:64] FLAG: --v="2" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197623 2569 flags.go:64] FLAG: --version="false" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197629 2569 flags.go:64] FLAG: --vmodule="" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197633 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.197637 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197749 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:15:35.416465 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197752 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197755 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197758 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197762 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197766 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197769 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197772 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197775 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197778 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197780 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197783 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197788 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197790 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197793 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197796 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197799 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197801 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197804 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197807 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:15:35.424054 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197813 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197816 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197819 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197822 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197824 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197827 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197830 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197832 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197835 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197837 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197840 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197843 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197845 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197848 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197850 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197853 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197855 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197858 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197860 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197863 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:15:35.501659 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197865 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197867 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197870 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197872 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197876 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197879 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197881 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197885 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197888 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197891 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197893 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197896 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197900 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197903 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197905 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197908 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197910 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197913 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197916 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197918 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:15:35.502671 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197921 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197923 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197926 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197929 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197932 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197934 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197937 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197939 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197942 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197944 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197947 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197949 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197952 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197955 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197958 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197960 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197964 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197966 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197969 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197971 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:15:35.503562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197973 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197976 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197978 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197981 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197985 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197988 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197990 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197993 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197996 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.197998 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.198001 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.198003 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.198006 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.198008 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.198011 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.198013 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.198016 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.198019 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:15:35.504463 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.198021 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.198638 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.207369 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.207384 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.207404 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.207410 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207465 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207470 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207474 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207477 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207480 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207483 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207487 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207490 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207493 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:15:35.505362 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207496 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207499 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207502 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207505 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207508 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207510 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207513 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207515 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207518 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207520 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207523 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207525 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207528 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207530 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207533 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207535 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207538 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207541 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207543 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207546 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:15:35.506044 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207548 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207553 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207557 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207560 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207567 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207572 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207575 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207577 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207580 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207583 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207586 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207588 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207591 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207594 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207596 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207599 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207601 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207604 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207607 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:15:35.506987 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207610 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207612 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207615 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207617 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207620 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207622 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207625 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207627 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207630 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207632 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207635 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207637 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207640 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207644 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207647 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207651 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207653 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207658 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207662 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:15:35.507892 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207665 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207667 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207670 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207672 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207675 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207677 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207680 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207682 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207685 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207688 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207690 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207693 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207696 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207698 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207701 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207703 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207706 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207708 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207711 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207713 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:15:35.508779 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207716 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207720 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207723 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207726 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207729 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207732 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207735 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207737 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207740 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207743 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207747 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207751 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.207756 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207868 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:15:35.509698 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207873 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207876 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207879 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207882 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207884 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207887 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207890 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207893 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207895 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207899 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207901 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207905 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207907 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207910 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207912 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207915 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207917 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207920 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207923 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207925 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:15:35.510377 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207928 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207930 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207933 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207935 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207939 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207942 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207945 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207948 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207951 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207956 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207959 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207961 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207964 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207967 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207969 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207972 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207975 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207977 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207980 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:15:35.511477 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207982 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207985 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207988 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207990 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207993 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207996 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.207999 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208001 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208004 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208008 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208011 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208014 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208017 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208019 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208022 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208024 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208027 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208030 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208032 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:15:35.512365 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208034 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208037 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208040 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208045 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208047 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208050 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208052 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208055 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208058 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208060 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208063 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208065 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208068 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208071 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208073 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208076 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208078 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208081 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208083 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208086 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:15:35.513471 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208089 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208092 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208094 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208096 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208099 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208103 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208106 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208109 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208112 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208114 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208117 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208120 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208123 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208125 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208128 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208133 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208136 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208138 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208141 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:15:35.514345 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:35.208160 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.208166 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.208270 2569 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.211981 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.213372 2569 server.go:1019] "Starting client certificate rotation" Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.213472 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.214179 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.216232 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.220592 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.233208 2569 log.go:25] "Validated CRI v1 runtime API" Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.236263 2569 log.go:25] "Validated CRI v1 image API" Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.237163 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 14:15:35.515454 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.242411 2569 fs.go:135] Filesystem UUIDs: map[098912aa-c878-4450-b5b5-ced6ebccc4c7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9ef8573f-6c2a-4b84-b110-75702ffa0502:/dev/nvme0n1p4] Jul 02 14:15:35.516105 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.242426 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 14:15:35.516105 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.246298 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.250160 2569 manager.go:216] Machine: {Timestamp:2026-07-02 14:15:35.248697708 +0000 UTC m=+0.368080520 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100855 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec242781fdeed5f83ad2f5bcfac7c20b SystemUUID:ec242781-fdee-d5f8-3ad2-f5bcfac7c20b BootID:82d17dfc-23ae-4957-ba0c-b8a59df15574 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ee:e1:18:71:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ee:e1:18:71:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:ea:44:89:44:2e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.250259 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.250341 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.253541 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.253567 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-134.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.254938 2569 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.254948 2569 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.254956 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.255688 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.256678 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.261231 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.261364 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.261393 2569 kubelet.go:486] "Attempting to sync node with API server" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.261408 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.261422 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.261431 2569 kubelet.go:398] "Adding apiserver pod source" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.261441 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 14:15:35.516544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.261859 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.261873 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.264599 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.266115 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.266142 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.266328 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.266339 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.266346 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.266352 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.266360 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.266366 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.268242 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.268254 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.268265 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.268274 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.268287 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.268300 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.269122 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.269130 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.272363 2569 server.go:1268] "Started kubelet" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.272464 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.272605 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.272678 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.272689 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.272699 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.272825 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-134.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.273877 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.274900 2569 server.go:310] "Adding debug handlers to kubelet server" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.278463 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.278747 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-134.ec2.internal.18be7ea1a8d010fa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-134.ec2.internal,UID:ip-10-0-138-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-134.ec2.internal,},FirstTimestamp:2026-07-02 14:15:35.272337658 +0000 UTC m=+0.391720469,LastTimestamp:2026-07-02 14:15:35.272337658 +0000 UTC m=+0.391720469,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-134.ec2.internal,}" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.280790 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.280806 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.280936 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.281357 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.281360 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.281369 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.281659 2569 reconstruct.go:97] "Volume reconstruction finished" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.281682 2569 reconciler.go:29] "Reconciler: start to sync state" Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.282245 2569 factory.go:55] Registering systemd factory Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.282262 2569 factory.go:223] Registration of the systemd container factory successfully Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.282595 2569 factory.go:153] Registering CRI-O factory Jul 02 14:15:35.519595 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.282613 2569 factory.go:223] Registration of the crio container factory successfully Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.282674 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.282701 2569 factory.go:103] Registering Raw factory Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.282718 2569 manager.go:1196] Started watching for new ooms in manager Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.283266 2569 manager.go:349] Starting recovery of all containers Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.290921 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.291062 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.294562 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qfqc7" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.296697 2569 manager.go:354] Recovery completed Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.300398 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qfqc7" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.300697 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.302556 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.302584 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.302598 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.303120 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.303133 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.303129 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.303241 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.304018 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.304043 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.304058 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.304308 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-134.ec2.internal.18be7ea1aa9d595e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-134.ec2.internal,UID:ip-10-0-138-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-134.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-134.ec2.internal,},FirstTimestamp:2026-07-02 14:15:35.302568286 +0000 UTC m=+0.421951099,LastTimestamp:2026-07-02 14:15:35.302568286 +0000 UTC m=+0.421951099,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-134.ec2.internal,}" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.305573 2569 policy_none.go:49] "None policy: Start" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.305584 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.305594 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.307466 2569 policy_none.go:47] "Start" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.346863 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.348000 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.348040 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.348067 2569 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.348109 2569 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.350435 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.381979 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.449174 2569 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.451304 2569 manager.go:337] "Starting Device Plugin manager" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.451600 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.451613 2569 server.go:87] "Starting device plugin registration server" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.451840 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.451850 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 14:15:35.521358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.452169 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 14:15:35.529369 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.452430 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 14:15:35.529369 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.452440 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 14:15:35.529369 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.454044 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 14:15:35.529369 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.454096 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:35.529369 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.496374 2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-134.ec2.internal\" not found" node="ip-10-0-138-134.ec2.internal" Jul 02 14:15:35.598288 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.552490 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:15:35.598288 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.553507 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:15:35.598288 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.553535 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:15:35.598288 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.553546 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:15:35.598288 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.553571 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-134.ec2.internal" Jul 02 14:15:35.598288 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.558989 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-134.ec2.internal" Jul 02 14:15:35.598288 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.559009 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-134.ec2.internal\": node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:35.598288 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.591544 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.650238 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal"] Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.650321 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.652125 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.652179 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.652193 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.654520 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.654591 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.654619 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.655251 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.655276 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.655286 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.655252 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.655352 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.655364 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.657382 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.657402 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.658098 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.658115 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:15:35.674615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.658124 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:15:35.741292 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.685706 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/15d4eff56ed4ced4d7d17ab47ab02ee1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-134.ec2.internal\" (UID: \"15d4eff56ed4ced4d7d17ab47ab02ee1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" Jul 02 14:15:35.741292 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.685734 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e618edf4acfd31e812daff3a0b9a6168-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"e618edf4acfd31e812daff3a0b9a6168\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" Jul 02 14:15:35.741292 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.685752 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e618edf4acfd31e812daff3a0b9a6168-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"e618edf4acfd31e812daff3a0b9a6168\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" Jul 02 14:15:35.741292 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.692033 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:37.044755 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.786226 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/15d4eff56ed4ced4d7d17ab47ab02ee1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-134.ec2.internal\" (UID: \"15d4eff56ed4ced4d7d17ab47ab02ee1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.044755 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.786253 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e618edf4acfd31e812daff3a0b9a6168-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"e618edf4acfd31e812daff3a0b9a6168\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.044755 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.786270 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e618edf4acfd31e812daff3a0b9a6168-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"e618edf4acfd31e812daff3a0b9a6168\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.044755 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.786310 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e618edf4acfd31e812daff3a0b9a6168-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"e618edf4acfd31e812daff3a0b9a6168\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.044755 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.786333 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/15d4eff56ed4ced4d7d17ab47ab02ee1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-134.ec2.internal\" (UID: \"15d4eff56ed4ced4d7d17ab47ab02ee1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.044755 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:35.786333 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e618edf4acfd31e812daff3a0b9a6168-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"e618edf4acfd31e812daff3a0b9a6168\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.044755 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:35.792315 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:37.178487 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:37.137123 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:37.178487 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:37.144783 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-134.ec2.internal\" not found" node="ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.178487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.148738 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.178487 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:37.149257 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-134.ec2.internal\" not found" node="ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.178487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.152299 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.253051 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:37.237492 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:37.394358 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:37.338022 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:37.405432 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.404773 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:15:37.477679 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:37.438500 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:37.477679 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.456581 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 14:15:37.477679 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.456736 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 14:15:37.477679 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.456759 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 14:15:37.572985 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:37.538695 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:37.572985 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.550274 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 14:10:36 +0000 UTC" deadline="2028-02-01 10:28:19.896629818 +0000 UTC" Jul 02 14:15:37.572985 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.550294 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13892h12m42.346338122s" Jul 02 14:15:37.680059 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:37.639273 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found" Jul 02 14:15:37.680059 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.674061 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:15:37.773066 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.726255 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.773066 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.736839 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 14:15:37.773066 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.737784 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" Jul 02 14:15:37.773066 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.748180 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 14:15:37.872543 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:37.872513 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d4eff56ed4ced4d7d17ab47ab02ee1.slice/crio-dfbe1b3895b6ca3c766a5a296dc1142593d36f0a547f33628e6850ac03653a70 WatchSource:0}: Error finding container dfbe1b3895b6ca3c766a5a296dc1142593d36f0a547f33628e6850ac03653a70: Status 404 returned error can't find the container with id dfbe1b3895b6ca3c766a5a296dc1142593d36f0a547f33628e6850ac03653a70 Jul 02 14:15:37.872991 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:37.872969 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode618edf4acfd31e812daff3a0b9a6168.slice/crio-9d72a43ffad94a546b7daab1d7cb9056e1da63225dab7956fca22e53930a3825 WatchSource:0}: Error finding container 9d72a43ffad94a546b7daab1d7cb9056e1da63225dab7956fca22e53930a3825: Status 404 returned error can't find the container with id 9d72a43ffad94a546b7daab1d7cb9056e1da63225dab7956fca22e53930a3825 Jul 02 14:15:37.877171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:37.877154 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 14:15:38.068985 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.068906 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:15:38.508287 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.507977 2569 apiserver.go:52] "Watching apiserver" Jul 02 14:15:38.519483 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.518523 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 14:15:38.519483 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.519067 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-dcshg","openshift-cluster-node-tuning-operator/tuned-thntg","openshift-image-registry/node-ca-8jkcr","openshift-multus/multus-crjcx","openshift-multus/network-metrics-daemon-9z5n8","openshift-ovn-kubernetes/ovnkube-node-74t6q","kube-system/global-pull-secret-syncer-df6mz","kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7","openshift-dns/node-resolver-mf4f7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal","openshift-multus/multus-additional-cni-plugins-zhddh","openshift-network-diagnostics/network-check-target-rzbpn","openshift-network-operator/iptables-alerter-kr8t2"] Jul 02 14:15:38.521865 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.521539 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:38.521865 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.521634 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:38.523759 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.523736 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:38.523857 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.523805 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:38.527908 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.527886 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 14:15:38.530118 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.528542 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.531509 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.531034 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:38.531509 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.531108 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:38.531509 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.531312 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 14:15:38.531509 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.531338 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 14:15:38.531849 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.531799 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.535137 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.533477 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dqqvk\"" Jul 02 14:15:38.535137 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.533675 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 14:15:38.535137 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.533862 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.535322 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.535282 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.535718 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.535731 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536280 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536472 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536678 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-96z5j\"" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536836 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cc01e91-17e6-465e-b77d-acceac9e4a27-host\") pod \"node-ca-8jkcr\" (UID: \"8cc01e91-17e6-465e-b77d-acceac9e4a27\") " pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536854 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-run-k8s-cni-cncf-io\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536873 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-multus-socket-dir-parent\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536890 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-run-netns\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536907 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-run-multus-certs\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536921 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536939 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-etc-kubernetes\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536969 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nhzw\" (UniqueName: \"kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw\") pod \"network-check-target-rzbpn\" (UID: \"b5a88716-1c0d-47f1-95b5-50c0b04d89c1\") " pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.536993 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8cc01e91-17e6-465e-b77d-acceac9e4a27-serviceca\") pod \"node-ca-8jkcr\" (UID: \"8cc01e91-17e6-465e-b77d-acceac9e4a27\") " pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.537037 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvn7g\" (UniqueName: \"kubernetes.io/projected/8cc01e91-17e6-465e-b77d-acceac9e4a27-kube-api-access-fvn7g\") pod \"node-ca-8jkcr\" (UID: \"8cc01e91-17e6-465e-b77d-acceac9e4a27\") " pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.537104 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-multus-conf-dir\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.537138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.537147 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b33b423-4de4-41fa-85de-d5ae343c63aa-dbus\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:38.537966 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.537191 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-system-cni-dir\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.537966 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.537227 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-multus-cni-dir\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.537966 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.537254 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-cnibin\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.537966 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.537300 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-os-release\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.537966 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.537325 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pmpk\" (UniqueName: \"kubernetes.io/projected/4948db6d-308b-4367-927b-47c82a9ad85d-kube-api-access-4pmpk\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.537361 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjtq8\" (UniqueName: \"kubernetes.io/projected/74858031-b9e7-4b68-83b7-9cd8e9b22824-kube-api-access-hjtq8\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.539542 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4948db6d-308b-4367-927b-47c82a9ad85d-cni-binary-copy\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.539593 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-var-lib-cni-bin\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.539651 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-hostroot\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.539685 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.540089 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.540520 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-var-lib-cni-multus\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.540562 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-var-lib-kubelet\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.540604 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4948db6d-308b-4367-927b-47c82a9ad85d-multus-daemon-config\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.540634 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b33b423-4de4-41fa-85de-d5ae343c63aa-kubelet-config\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:38.541031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.540876 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:15:38.541638 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.541558 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 14:15:38.541883 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.541865 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 14:15:38.541989 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.541971 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 14:15:38.542100 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.541865 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 14:15:38.542198 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.542183 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q4kdr\"" Jul 02 14:15:38.542323 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.542305 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5g7b4\"" Jul 02 14:15:38.542412 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.542397 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 14:15:38.542654 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.542637 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 14:15:38.542854 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.542837 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 14:15:38.543589 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.543571 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 14:15:38.544838 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.544820 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4sdrg\"" Jul 02 14:15:38.544957 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.544936 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 14:15:38.545127 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.545111 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 14:15:38.546467 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.546064 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.546467 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.546077 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 14:15:38.549001 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.548622 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 14:15:38.549001 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.548728 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pgl8n\"" Jul 02 14:15:38.549001 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.548838 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 14:15:38.549545 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.549526 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.552855 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.551995 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 14:15:38.552855 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.552187 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.552855 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.552317 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xlv8s\"" Jul 02 14:15:38.552855 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.552763 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 14:15:38.555280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.554522 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.555280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.554754 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 14:15:38.555280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.555092 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 14:15:38.555280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.555101 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7947n\"" Jul 02 14:15:38.555280 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.555142 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 14:15:38.557599 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.557110 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7bhd9\"" Jul 02 14:15:38.557599 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.557297 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 14:15:38.557599 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.557546 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 14:15:38.569990 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.569970 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f484l" Jul 02 14:15:38.579458 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.579267 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f484l" Jul 02 14:15:38.600308 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.600258 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" event={"ID":"15d4eff56ed4ced4d7d17ab47ab02ee1","Type":"ContainerStarted","Data":"dfbe1b3895b6ca3c766a5a296dc1142593d36f0a547f33628e6850ac03653a70"} Jul 02 14:15:38.602005 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.601978 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" event={"ID":"e618edf4acfd31e812daff3a0b9a6168","Type":"ContainerStarted","Data":"9d72a43ffad94a546b7daab1d7cb9056e1da63225dab7956fca22e53930a3825"} Jul 02 14:15:38.618848 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.618822 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:15:38.627142 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.627118 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 14:15:38.644089 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644055 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nnhp\" (UniqueName: \"kubernetes.io/projected/2b739ac8-5c98-4d5a-a3bf-133c98962b46-kube-api-access-2nnhp\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.644242 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644105 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2833a111-c1fc-4670-92e5-9dfacb75df80-agent-certs\") pod \"konnectivity-agent-dcshg\" (UID: \"2833a111-c1fc-4670-92e5-9dfacb75df80\") " pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:15:38.644242 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644132 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-sysconfig\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.644242 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644166 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-sysctl-d\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.644242 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644196 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-var-lib-kubelet\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.644242 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644221 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4948db6d-308b-4367-927b-47c82a9ad85d-multus-daemon-config\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644244 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1942e8f3-282c-427b-af88-9bf72887e1e6-iptables-alerter-script\") pod \"iptables-alerter-kr8t2\" (UID: \"1942e8f3-282c-427b-af88-9bf72887e1e6\") " pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644272 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-systemd-units\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644294 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-run-ovn\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644314 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-host\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644342 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4cgf\" (UniqueName: \"kubernetes.io/projected/673457cc-befb-4acf-a4cc-270c97edfc9c-kube-api-access-m4cgf\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644370 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b33b423-4de4-41fa-85de-d5ae343c63aa-kubelet-config\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644394 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644419 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-run-k8s-cni-cncf-io\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644458 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-run-systemd\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644483 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-var-lib-openvswitch\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644505 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb98cb35-0b85-40e4-9a88-54f73548fb3d-ovnkube-config\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644543 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-socket-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.644586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644583 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-multus-socket-dir-parent\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644609 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-run-netns\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644630 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-slash\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644660 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-etc-openvswitch\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644686 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644710 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-var-lib-kubelet\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644736 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d625f978-90bd-4f61-b0a2-644876bcf84d-tmp\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644759 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-cnibin\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644781 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-modprobe-d\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644804 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644829 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhzw\" (UniqueName: \"kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw\") pod \"network-check-target-rzbpn\" (UID: \"b5a88716-1c0d-47f1-95b5-50c0b04d89c1\") " pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644863 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvn7g\" (UniqueName: \"kubernetes.io/projected/8cc01e91-17e6-465e-b77d-acceac9e4a27-kube-api-access-fvn7g\") pod \"node-ca-8jkcr\" (UID: \"8cc01e91-17e6-465e-b77d-acceac9e4a27\") " pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-multus-conf-dir\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644922 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea0e1336-22b5-4c5c-b34d-e4a469f0bc60-hosts-file\") pod \"node-resolver-mf4f7\" (UID: \"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60\") " pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644947 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-system-cni-dir\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.644970 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-os-release\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.645064 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645009 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpxsk\" (UniqueName: \"kubernetes.io/projected/fb98cb35-0b85-40e4-9a88-54f73548fb3d-kube-api-access-rpxsk\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645033 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-sys\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645058 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-system-cni-dir\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645079 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-cnibin\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645103 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bskw9\" (UniqueName: \"kubernetes.io/projected/ea0e1336-22b5-4c5c-b34d-e4a469f0bc60-kube-api-access-bskw9\") pod \"node-resolver-mf4f7\" (UID: \"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60\") " pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645237 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-multus-socket-dir-parent\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645320 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b33b423-4de4-41fa-85de-d5ae343c63aa-kubelet-config\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645369 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-var-lib-kubelet\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645406 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-log-socket\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645433 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-systemd\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645476 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42g9j\" (UniqueName: \"kubernetes.io/projected/d625f978-90bd-4f61-b0a2-644876bcf84d-kube-api-access-42g9j\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645503 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-sys-fs\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645533 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjtq8\" (UniqueName: \"kubernetes.io/projected/74858031-b9e7-4b68-83b7-9cd8e9b22824-kube-api-access-hjtq8\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-var-lib-cni-bin\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645583 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-hostroot\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645624 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2833a111-c1fc-4670-92e5-9dfacb75df80-konnectivity-ca\") pod \"konnectivity-agent-dcshg\" (UID: \"2833a111-c1fc-4670-92e5-9dfacb75df80\") " pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645646 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:38.645918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645665 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-var-lib-cni-multus\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645694 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-cni-netd\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645723 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb98cb35-0b85-40e4-9a88-54f73548fb3d-ovnkube-script-lib\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645757 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-tuned\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645819 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-registration-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645859 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-run-netns\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645878 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645906 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-run-k8s-cni-cncf-io\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.645933 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cc01e91-17e6-465e-b77d-acceac9e4a27-host\") pod \"node-ca-8jkcr\" (UID: \"8cc01e91-17e6-465e-b77d-acceac9e4a27\") " pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.646003 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-system-cni-dir\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.646252 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea0e1336-22b5-4c5c-b34d-e4a469f0bc60-tmp-dir\") pod \"node-resolver-mf4f7\" (UID: \"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60\") " pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.646278 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cc01e91-17e6-465e-b77d-acceac9e4a27-host\") pod \"node-ca-8jkcr\" (UID: \"8cc01e91-17e6-465e-b77d-acceac9e4a27\") " pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.646296 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-run-openvswitch\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.646363 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-hostroot\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.646354 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-multus-conf-dir\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.646687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.646371 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-var-lib-cni-bin\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.647355 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.647019 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4948db6d-308b-4367-927b-47c82a9ad85d-multus-daemon-config\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.647355 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.647088 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-lib-modules\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.647355 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.647122 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-run-multus-certs\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.647355 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.647178 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b739ac8-5c98-4d5a-a3bf-133c98962b46-cni-binary-copy\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.647355 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.647308 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-var-lib-cni-multus\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.647594 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.647421 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:38.647651 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.647616 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret podName:5b33b423-4de4-41fa-85de-d5ae343c63aa nodeName:}" failed. No retries permitted until 2026-07-02 14:15:39.14758717 +0000 UTC m=+3.022670620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret") pod "global-pull-secret-syncer-df6mz" (UID: "5b33b423-4de4-41fa-85de-d5ae343c63aa") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:38.648143 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.648092 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-cnibin\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.648232 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.648153 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-host-run-multus-certs\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.648322 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.648294 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b739ac8-5c98-4d5a-a3bf-133c98962b46-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.648482 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.648437 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-kubelet\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.648690 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.648672 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-cni-bin\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.648757 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.648734 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:38.649004 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.648987 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:38.649160 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.649144 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs podName:74858031-b9e7-4b68-83b7-9cd8e9b22824 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:39.149117907 +0000 UTC m=+3.024201355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs") pod "network-metrics-daemon-9z5n8" (UID: "74858031-b9e7-4b68-83b7-9cd8e9b22824") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:38.649238 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649200 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-etc-kubernetes\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.649311 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649283 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1942e8f3-282c-427b-af88-9bf72887e1e6-host-slash\") pod \"iptables-alerter-kr8t2\" (UID: \"1942e8f3-282c-427b-af88-9bf72887e1e6\") " pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.649378 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649359 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b739ac8-5c98-4d5a-a3bf-133c98962b46-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.649463 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649421 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb98cb35-0b85-40e4-9a88-54f73548fb3d-ovn-node-metrics-cert\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.649524 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649478 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-kubernetes\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.649650 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649551 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-sysctl-conf\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.649650 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649579 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-run\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.649650 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649622 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8cc01e91-17e6-465e-b77d-acceac9e4a27-serviceca\") pod \"node-ca-8jkcr\" (UID: \"8cc01e91-17e6-465e-b77d-acceac9e4a27\") " pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.649825 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649654 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb98cb35-0b85-40e4-9a88-54f73548fb3d-env-overrides\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.649825 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649704 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-device-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.650069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649813 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b33b423-4de4-41fa-85de-d5ae343c63aa-dbus\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:38.650069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649863 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-multus-cni-dir\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.650069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.649930 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-os-release\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.650269 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650147 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pmpk\" (UniqueName: \"kubernetes.io/projected/4948db6d-308b-4367-927b-47c82a9ad85d-kube-api-access-4pmpk\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.650269 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650198 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s256q\" (UniqueName: \"kubernetes.io/projected/1942e8f3-282c-427b-af88-9bf72887e1e6-kube-api-access-s256q\") pod \"iptables-alerter-kr8t2\" (UID: \"1942e8f3-282c-427b-af88-9bf72887e1e6\") " pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.650269 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650229 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-run-netns\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.650269 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650229 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-os-release\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.650505 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650260 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-node-log\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.650505 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650277 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-etc-kubernetes\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.650505 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650316 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-run-ovn-kubernetes\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.650784 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650755 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8cc01e91-17e6-465e-b77d-acceac9e4a27-serviceca\") pod \"node-ca-8jkcr\" (UID: \"8cc01e91-17e6-465e-b77d-acceac9e4a27\") " pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.650841 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650800 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4948db6d-308b-4367-927b-47c82a9ad85d-cni-binary-copy\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.650841 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650805 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b33b423-4de4-41fa-85de-d5ae343c63aa-dbus\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:38.650949 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.650893 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4948db6d-308b-4367-927b-47c82a9ad85d-multus-cni-dir\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.651320 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.651303 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4948db6d-308b-4367-927b-47c82a9ad85d-cni-binary-copy\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.656324 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.656301 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:15:38.656443 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.656333 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:15:38.656443 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.656347 2569 projected.go:196] Error preparing data for projected volume kube-api-access-9nhzw for pod openshift-network-diagnostics/network-check-target-rzbpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:38.656443 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:38.656423 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw podName:b5a88716-1c0d-47f1-95b5-50c0b04d89c1 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:39.156406591 +0000 UTC m=+3.031490042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9nhzw" (UniqueName: "kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw") pod "network-check-target-rzbpn" (UID: "b5a88716-1c0d-47f1-95b5-50c0b04d89c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:38.656897 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.656851 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 14:15:38.661102 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.661044 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjtq8\" (UniqueName: \"kubernetes.io/projected/74858031-b9e7-4b68-83b7-9cd8e9b22824-kube-api-access-hjtq8\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:38.661102 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.661079 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvn7g\" (UniqueName: \"kubernetes.io/projected/8cc01e91-17e6-465e-b77d-acceac9e4a27-kube-api-access-fvn7g\") pod \"node-ca-8jkcr\" (UID: \"8cc01e91-17e6-465e-b77d-acceac9e4a27\") " pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.661629 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.661586 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pmpk\" (UniqueName: \"kubernetes.io/projected/4948db6d-308b-4367-927b-47c82a9ad85d-kube-api-access-4pmpk\") pod \"multus-crjcx\" (UID: \"4948db6d-308b-4367-927b-47c82a9ad85d\") " pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.751316 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751284 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea0e1336-22b5-4c5c-b34d-e4a469f0bc60-tmp-dir\") pod \"node-resolver-mf4f7\" (UID: \"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60\") " pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.751316 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751322 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-run-openvswitch\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751345 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-lib-modules\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751398 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-run-openvswitch\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751516 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b739ac8-5c98-4d5a-a3bf-133c98962b46-cni-binary-copy\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751561 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b739ac8-5c98-4d5a-a3bf-133c98962b46-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751601 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-kubelet\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751620 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-lib-modules\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751624 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-cni-bin\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751659 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-cni-bin\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751660 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea0e1336-22b5-4c5c-b34d-e4a469f0bc60-tmp-dir\") pod \"node-resolver-mf4f7\" (UID: \"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60\") " pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1942e8f3-282c-427b-af88-9bf72887e1e6-host-slash\") pod \"iptables-alerter-kr8t2\" (UID: \"1942e8f3-282c-427b-af88-9bf72887e1e6\") " pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751693 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-kubelet\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751711 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b739ac8-5c98-4d5a-a3bf-133c98962b46-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751801 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb98cb35-0b85-40e4-9a88-54f73548fb3d-ovn-node-metrics-cert\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751799 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1942e8f3-282c-427b-af88-9bf72887e1e6-host-slash\") pod \"iptables-alerter-kr8t2\" (UID: \"1942e8f3-282c-427b-af88-9bf72887e1e6\") " pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751861 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-kubernetes\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751881 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-sysctl-conf\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.752046 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751913 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-run\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751937 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-kubernetes\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751937 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb98cb35-0b85-40e4-9a88-54f73548fb3d-env-overrides\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.751974 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-device-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752181 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b739ac8-5c98-4d5a-a3bf-133c98962b46-cni-binary-copy\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752200 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s256q\" (UniqueName: \"kubernetes.io/projected/1942e8f3-282c-427b-af88-9bf72887e1e6-kube-api-access-s256q\") pod \"iptables-alerter-kr8t2\" (UID: \"1942e8f3-282c-427b-af88-9bf72887e1e6\") " pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752215 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-sysctl-conf\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752242 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-run\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752283 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b739ac8-5c98-4d5a-a3bf-133c98962b46-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752300 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-run-netns\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752326 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-node-log\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752336 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-run-netns\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752337 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b739ac8-5c98-4d5a-a3bf-133c98962b46-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752376 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb98cb35-0b85-40e4-9a88-54f73548fb3d-env-overrides\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752380 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-run-ovn-kubernetes\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752409 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-node-log\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752418 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-run-ovn-kubernetes\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.752853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752428 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-device-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752482 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nnhp\" (UniqueName: \"kubernetes.io/projected/2b739ac8-5c98-4d5a-a3bf-133c98962b46-kube-api-access-2nnhp\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752510 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2833a111-c1fc-4670-92e5-9dfacb75df80-agent-certs\") pod \"konnectivity-agent-dcshg\" (UID: \"2833a111-c1fc-4670-92e5-9dfacb75df80\") " pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752534 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-sysconfig\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752557 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-sysctl-d\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752592 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1942e8f3-282c-427b-af88-9bf72887e1e6-iptables-alerter-script\") pod \"iptables-alerter-kr8t2\" (UID: \"1942e8f3-282c-427b-af88-9bf72887e1e6\") " pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752618 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-systemd-units\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752637 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-sysconfig\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752650 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-run-ovn\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752675 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-host\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752698 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4cgf\" (UniqueName: \"kubernetes.io/projected/673457cc-befb-4acf-a4cc-270c97edfc9c-kube-api-access-m4cgf\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752721 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752739 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-sysctl-d\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752751 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-run-systemd\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752778 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-var-lib-openvswitch\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752797 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-host\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752821 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb98cb35-0b85-40e4-9a88-54f73548fb3d-ovnkube-config\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.753657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752842 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-systemd-units\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752848 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-socket-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752877 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-slash\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752882 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-run-ovn\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752903 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-etc-openvswitch\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752924 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-var-lib-openvswitch\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752930 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.752983 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753024 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-etc-openvswitch\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753060 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-slash\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753075 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-run-systemd\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753078 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-socket-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753076 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753097 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-var-lib-kubelet\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753145 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d625f978-90bd-4f61-b0a2-644876bcf84d-tmp\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753148 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-var-lib-kubelet\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753171 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-cnibin\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.754460 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753194 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1942e8f3-282c-427b-af88-9bf72887e1e6-iptables-alerter-script\") pod \"iptables-alerter-kr8t2\" (UID: \"1942e8f3-282c-427b-af88-9bf72887e1e6\") " pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753198 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-modprobe-d\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753242 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753267 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-modprobe-d\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753281 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea0e1336-22b5-4c5c-b34d-e4a469f0bc60-hosts-file\") pod \"node-resolver-mf4f7\" (UID: \"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60\") " pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753307 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-cnibin\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753305 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-system-cni-dir\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753350 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753347 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-os-release\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753383 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpxsk\" (UniqueName: \"kubernetes.io/projected/fb98cb35-0b85-40e4-9a88-54f73548fb3d-kube-api-access-rpxsk\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753398 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea0e1336-22b5-4c5c-b34d-e4a469f0bc60-hosts-file\") pod \"node-resolver-mf4f7\" (UID: \"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60\") " pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753405 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-sys\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753443 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bskw9\" (UniqueName: \"kubernetes.io/projected/ea0e1336-22b5-4c5c-b34d-e4a469f0bc60-kube-api-access-bskw9\") pod \"node-resolver-mf4f7\" (UID: \"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60\") " pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753477 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-sys\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753488 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-log-socket\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753516 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-systemd\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753523 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-os-release\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.755049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753541 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42g9j\" (UniqueName: \"kubernetes.io/projected/d625f978-90bd-4f61-b0a2-644876bcf84d-kube-api-access-42g9j\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753565 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb98cb35-0b85-40e4-9a88-54f73548fb3d-ovnkube-config\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753621 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-systemd\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753627 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-sys-fs\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753567 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-sys-fs\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2833a111-c1fc-4670-92e5-9dfacb75df80-konnectivity-ca\") pod \"konnectivity-agent-dcshg\" (UID: \"2833a111-c1fc-4670-92e5-9dfacb75df80\") " pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753681 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-log-socket\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753566 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b739ac8-5c98-4d5a-a3bf-133c98962b46-system-cni-dir\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753717 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-cni-netd\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753747 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb98cb35-0b85-40e4-9a88-54f73548fb3d-ovnkube-script-lib\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753771 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-tuned\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753794 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-registration-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753819 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753904 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753952 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/673457cc-befb-4acf-a4cc-270c97edfc9c-registration-dir\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.753989 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb98cb35-0b85-40e4-9a88-54f73548fb3d-host-cni-netd\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.754228 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2833a111-c1fc-4670-92e5-9dfacb75df80-konnectivity-ca\") pod \"konnectivity-agent-dcshg\" (UID: \"2833a111-c1fc-4670-92e5-9dfacb75df80\") " pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:15:38.755691 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.754380 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb98cb35-0b85-40e4-9a88-54f73548fb3d-ovnkube-script-lib\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.756438 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.756416 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d625f978-90bd-4f61-b0a2-644876bcf84d-etc-tuned\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.756829 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.756807 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb98cb35-0b85-40e4-9a88-54f73548fb3d-ovn-node-metrics-cert\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.758399 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.758343 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2833a111-c1fc-4670-92e5-9dfacb75df80-agent-certs\") pod \"konnectivity-agent-dcshg\" (UID: \"2833a111-c1fc-4670-92e5-9dfacb75df80\") " pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:15:38.758783 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.758753 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d625f978-90bd-4f61-b0a2-644876bcf84d-tmp\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.766799 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.766750 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nnhp\" (UniqueName: \"kubernetes.io/projected/2b739ac8-5c98-4d5a-a3bf-133c98962b46-kube-api-access-2nnhp\") pod \"multus-additional-cni-plugins-zhddh\" (UID: \"2b739ac8-5c98-4d5a-a3bf-133c98962b46\") " pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:38.767537 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.767511 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bskw9\" (UniqueName: \"kubernetes.io/projected/ea0e1336-22b5-4c5c-b34d-e4a469f0bc60-kube-api-access-bskw9\") pod \"node-resolver-mf4f7\" (UID: \"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60\") " pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.767611 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.767512 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s256q\" (UniqueName: \"kubernetes.io/projected/1942e8f3-282c-427b-af88-9bf72887e1e6-kube-api-access-s256q\") pod \"iptables-alerter-kr8t2\" (UID: \"1942e8f3-282c-427b-af88-9bf72887e1e6\") " pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.767647 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.767623 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpxsk\" (UniqueName: \"kubernetes.io/projected/fb98cb35-0b85-40e4-9a88-54f73548fb3d-kube-api-access-rpxsk\") pod \"ovnkube-node-74t6q\" (UID: \"fb98cb35-0b85-40e4-9a88-54f73548fb3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.769143 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.769121 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42g9j\" (UniqueName: \"kubernetes.io/projected/d625f978-90bd-4f61-b0a2-644876bcf84d-kube-api-access-42g9j\") pod \"tuned-thntg\" (UID: \"d625f978-90bd-4f61-b0a2-644876bcf84d\") " pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.769565 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.769545 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4cgf\" (UniqueName: \"kubernetes.io/projected/673457cc-befb-4acf-a4cc-270c97edfc9c-kube-api-access-m4cgf\") pod \"aws-ebs-csi-driver-node-dmtv7\" (UID: \"673457cc-befb-4acf-a4cc-270c97edfc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.854267 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.854235 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8jkcr" Jul 02 14:15:38.860964 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.860942 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-crjcx" Jul 02 14:15:38.873627 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.873606 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:15:38.877321 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.877303 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" Jul 02 14:15:38.884862 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.884847 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:15:38.896421 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.896397 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-thntg" Jul 02 14:15:38.902053 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.902033 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mf4f7" Jul 02 14:15:38.911587 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.911566 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kr8t2" Jul 02 14:15:38.919090 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:38.919073 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zhddh" Jul 02 14:15:39.155974 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:39.155888 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:39.155974 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:39.155933 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:39.156161 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:39.156053 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:39.156161 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:39.156060 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:39.156161 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:39.156122 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs podName:74858031-b9e7-4b68-83b7-9cd8e9b22824 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:40.156103358 +0000 UTC m=+4.031186814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs") pod "network-metrics-daemon-9z5n8" (UID: "74858031-b9e7-4b68-83b7-9cd8e9b22824") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:39.156161 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:39.156142 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret podName:5b33b423-4de4-41fa-85de-d5ae343c63aa nodeName:}" failed. No retries permitted until 2026-07-02 14:15:40.156131652 +0000 UTC m=+4.031215090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret") pod "global-pull-secret-syncer-df6mz" (UID: "5b33b423-4de4-41fa-85de-d5ae343c63aa") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:39.256494 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:39.256457 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhzw\" (UniqueName: \"kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw\") pod \"network-check-target-rzbpn\" (UID: \"b5a88716-1c0d-47f1-95b5-50c0b04d89c1\") " pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:39.256682 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:39.256589 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:15:39.256682 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:39.256620 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:15:39.256682 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:39.256634 2569 projected.go:196] Error preparing data for projected volume kube-api-access-9nhzw for pod openshift-network-diagnostics/network-check-target-rzbpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:39.256831 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:39.256686 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw podName:b5a88716-1c0d-47f1-95b5-50c0b04d89c1 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:40.256672602 +0000 UTC m=+4.131756034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9nhzw" (UniqueName: "kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw") pod "network-check-target-rzbpn" (UID: "b5a88716-1c0d-47f1-95b5-50c0b04d89c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:39.582560 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:39.582460 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 14:10:38 +0000 UTC" deadline="2028-04-19 03:45:26.739399436 +0000 UTC" Jul 02 14:15:39.582560 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:39.582509 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15757h29m47.156893801s" Jul 02 14:15:39.792996 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:39.792961 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2833a111_c1fc_4670_92e5_9dfacb75df80.slice/crio-d7aeee47fd01789a92eb812df32a38d345920f49713feeccf324507c22fb847b WatchSource:0}: Error finding container d7aeee47fd01789a92eb812df32a38d345920f49713feeccf324507c22fb847b: Status 404 returned error can't find the container with id d7aeee47fd01789a92eb812df32a38d345920f49713feeccf324507c22fb847b Jul 02 14:15:39.794846 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:39.794822 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb98cb35_0b85_40e4_9a88_54f73548fb3d.slice/crio-a9880a7ae03b67aab9dab0fb22889dd02ccb219815c4d429d7733b454fc6c34d WatchSource:0}: Error finding container a9880a7ae03b67aab9dab0fb22889dd02ccb219815c4d429d7733b454fc6c34d: Status 404 returned error can't find the container with id a9880a7ae03b67aab9dab0fb22889dd02ccb219815c4d429d7733b454fc6c34d Jul 02 14:15:39.795876 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:39.795850 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673457cc_befb_4acf_a4cc_270c97edfc9c.slice/crio-24e00bc377e349507ce064d33cdc67ea6a7137e61282fa7d6f5bcdc52d58c243 WatchSource:0}: Error finding container 24e00bc377e349507ce064d33cdc67ea6a7137e61282fa7d6f5bcdc52d58c243: Status 404 returned error can't find the container with id 24e00bc377e349507ce064d33cdc67ea6a7137e61282fa7d6f5bcdc52d58c243 Jul 02 14:15:39.798737 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:39.798646 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd625f978_90bd_4f61_b0a2_644876bcf84d.slice/crio-54c3bb2334af7c460a7fc378cdad96590d4b250b2edd3424e9968150645e86fb WatchSource:0}: Error finding container 54c3bb2334af7c460a7fc378cdad96590d4b250b2edd3424e9968150645e86fb: Status 404 returned error can't find the container with id 54c3bb2334af7c460a7fc378cdad96590d4b250b2edd3424e9968150645e86fb Jul 02 14:15:39.799728 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:39.799707 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea0e1336_22b5_4c5c_b34d_e4a469f0bc60.slice/crio-b4e1d450d2f34c34b3d57a528dabd7d16cfa013516cedb71f2d5ea9a89ce3b8f WatchSource:0}: Error finding container b4e1d450d2f34c34b3d57a528dabd7d16cfa013516cedb71f2d5ea9a89ce3b8f: Status 404 returned error can't find the container with id b4e1d450d2f34c34b3d57a528dabd7d16cfa013516cedb71f2d5ea9a89ce3b8f Jul 02 14:15:39.800483 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:39.800362 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4948db6d_308b_4367_927b_47c82a9ad85d.slice/crio-a4cf61a3ca03f651e56a5ed02929a9ba2cb20978ddb4f0826d9bb607be829c69 WatchSource:0}: Error finding container a4cf61a3ca03f651e56a5ed02929a9ba2cb20978ddb4f0826d9bb607be829c69: Status 404 returned error can't find the container with id a4cf61a3ca03f651e56a5ed02929a9ba2cb20978ddb4f0826d9bb607be829c69 Jul 02 14:15:39.802604 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:15:39.802505 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1942e8f3_282c_427b_af88_9bf72887e1e6.slice/crio-f902a797789aaf78c679876dce85e33df3d7ca235d333398d83134f9cf06b0bb WatchSource:0}: Error finding container f902a797789aaf78c679876dce85e33df3d7ca235d333398d83134f9cf06b0bb: Status 404 returned error can't find the container with id f902a797789aaf78c679876dce85e33df3d7ca235d333398d83134f9cf06b0bb Jul 02 14:15:40.163122 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.162846 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:40.163122 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.163071 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:40.163122 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.163015 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:40.163363 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.163170 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret podName:5b33b423-4de4-41fa-85de-d5ae343c63aa nodeName:}" failed. No retries permitted until 2026-07-02 14:15:42.163151132 +0000 UTC m=+6.038234581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret") pod "global-pull-secret-syncer-df6mz" (UID: "5b33b423-4de4-41fa-85de-d5ae343c63aa") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:40.163363 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.163214 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:40.163363 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.163270 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs podName:74858031-b9e7-4b68-83b7-9cd8e9b22824 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:42.163255333 +0000 UTC m=+6.038338768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs") pod "network-metrics-daemon-9z5n8" (UID: "74858031-b9e7-4b68-83b7-9cd8e9b22824") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:40.264274 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.264240 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhzw\" (UniqueName: \"kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw\") pod \"network-check-target-rzbpn\" (UID: \"b5a88716-1c0d-47f1-95b5-50c0b04d89c1\") " pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:40.264471 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.264394 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:15:40.264471 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.264408 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:15:40.264471 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.264416 2569 projected.go:196] Error preparing data for projected volume kube-api-access-9nhzw for pod openshift-network-diagnostics/network-check-target-rzbpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:40.264631 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.264484 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw podName:b5a88716-1c0d-47f1-95b5-50c0b04d89c1 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:42.264466476 +0000 UTC m=+6.139549910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9nhzw" (UniqueName: "kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw") pod "network-check-target-rzbpn" (UID: "b5a88716-1c0d-47f1-95b5-50c0b04d89c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:40.484157 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.484074 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:15:40.583904 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.583637 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 14:10:38 +0000 UTC" deadline="2027-12-09 21:27:26.099754801 +0000 UTC" Jul 02 14:15:40.583904 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.583678 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12607h11m45.516081356s" Jul 02 14:15:40.597520 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.596522 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:40.597520 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.596706 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:40.597520 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.596862 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:40.597520 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.596943 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:40.597520 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.597009 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:40.597520 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:40.597091 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:40.620664 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.620305 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" event={"ID":"15d4eff56ed4ced4d7d17ab47ab02ee1","Type":"ContainerStarted","Data":"c7c278f7b21ba1e67bed124f242255ec9578556771e52f1793ee484c8ebcfcba"} Jul 02 14:15:40.631465 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.631424 2569 generic.go:356] "Generic (PLEG): container finished" podID="e618edf4acfd31e812daff3a0b9a6168" containerID="b0fc797aa913df45fc8e54e37b2101e2b39f22b36f4fbc50e7a7457662d2b564" exitCode=0 Jul 02 14:15:40.631862 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.631562 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" event={"ID":"e618edf4acfd31e812daff3a0b9a6168","Type":"ContainerDied","Data":"b0fc797aa913df45fc8e54e37b2101e2b39f22b36f4fbc50e7a7457662d2b564"} Jul 02 14:15:40.646255 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.646149 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8jkcr" event={"ID":"8cc01e91-17e6-465e-b77d-acceac9e4a27","Type":"ContainerStarted","Data":"0c81eb3ba58f2c58b8c9fdb94df173b080a1f54890cb8ccbb4acf9a18fd35b6e"} Jul 02 14:15:40.648062 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.648002 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kr8t2" event={"ID":"1942e8f3-282c-427b-af88-9bf72887e1e6","Type":"ContainerStarted","Data":"f902a797789aaf78c679876dce85e33df3d7ca235d333398d83134f9cf06b0bb"} Jul 02 14:15:40.663188 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.662873 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" podStartSLOduration=3.662856534 podStartE2EDuration="3.662856534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:15:40.6387993 +0000 UTC m=+4.513882966" watchObservedRunningTime="2026-07-02 14:15:40.662856534 +0000 UTC m=+4.537939990" Jul 02 14:15:40.666259 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.666148 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mf4f7" event={"ID":"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60","Type":"ContainerStarted","Data":"b4e1d450d2f34c34b3d57a528dabd7d16cfa013516cedb71f2d5ea9a89ce3b8f"} Jul 02 14:15:40.694714 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.694668 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-thntg" event={"ID":"d625f978-90bd-4f61-b0a2-644876bcf84d","Type":"ContainerStarted","Data":"54c3bb2334af7c460a7fc378cdad96590d4b250b2edd3424e9968150645e86fb"} Jul 02 14:15:40.697633 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.697598 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dcshg" event={"ID":"2833a111-c1fc-4670-92e5-9dfacb75df80","Type":"ContainerStarted","Data":"d7aeee47fd01789a92eb812df32a38d345920f49713feeccf324507c22fb847b"} Jul 02 14:15:40.701842 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.701620 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhddh" event={"ID":"2b739ac8-5c98-4d5a-a3bf-133c98962b46","Type":"ContainerStarted","Data":"05c7b0ecbadaac2692a63f569404d39456a52f0353a275ce7bc7ba33740b8163"} Jul 02 14:15:40.703552 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.703527 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-crjcx" event={"ID":"4948db6d-308b-4367-927b-47c82a9ad85d","Type":"ContainerStarted","Data":"a4cf61a3ca03f651e56a5ed02929a9ba2cb20978ddb4f0826d9bb607be829c69"} Jul 02 14:15:40.709102 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.709074 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" event={"ID":"673457cc-befb-4acf-a4cc-270c97edfc9c","Type":"ContainerStarted","Data":"24e00bc377e349507ce064d33cdc67ea6a7137e61282fa7d6f5bcdc52d58c243"} Jul 02 14:15:40.712937 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:40.712909 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" event={"ID":"fb98cb35-0b85-40e4-9a88-54f73548fb3d","Type":"ContainerStarted","Data":"a9880a7ae03b67aab9dab0fb22889dd02ccb219815c4d429d7733b454fc6c34d"} Jul 02 14:15:41.730995 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:41.730962 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" event={"ID":"e618edf4acfd31e812daff3a0b9a6168","Type":"ContainerStarted","Data":"84388d00c426c5209521ef5bfbab5fa880b7b1f9448f7a28b574cb2b6c96065f"} Jul 02 14:15:42.185514 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:42.180654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:42.185514 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:42.180711 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:42.185514 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.180863 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:42.185514 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.180926 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs podName:74858031-b9e7-4b68-83b7-9cd8e9b22824 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:46.180906037 +0000 UTC m=+10.055989473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs") pod "network-metrics-daemon-9z5n8" (UID: "74858031-b9e7-4b68-83b7-9cd8e9b22824") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:42.185514 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.181478 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:42.185514 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.181535 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret podName:5b33b423-4de4-41fa-85de-d5ae343c63aa nodeName:}" failed. No retries permitted until 2026-07-02 14:15:46.181518041 +0000 UTC m=+10.056601480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret") pod "global-pull-secret-syncer-df6mz" (UID: "5b33b423-4de4-41fa-85de-d5ae343c63aa") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:42.282069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:42.282031 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhzw\" (UniqueName: \"kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw\") pod \"network-check-target-rzbpn\" (UID: \"b5a88716-1c0d-47f1-95b5-50c0b04d89c1\") " pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:42.282306 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.282287 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:15:42.282370 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.282318 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:15:42.282370 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.282331 2569 projected.go:196] Error preparing data for projected volume kube-api-access-9nhzw for pod openshift-network-diagnostics/network-check-target-rzbpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:42.282492 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.282396 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw podName:b5a88716-1c0d-47f1-95b5-50c0b04d89c1 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:46.282377808 +0000 UTC m=+10.157461245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9nhzw" (UniqueName: "kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw") pod "network-check-target-rzbpn" (UID: "b5a88716-1c0d-47f1-95b5-50c0b04d89c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:42.594688 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:42.593939 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:42.594688 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.594083 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:42.594688 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:42.594235 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:42.594688 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.594346 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:42.594688 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:42.594400 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:42.594688 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:42.594482 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:44.593625 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:44.593584 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:44.593625 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:44.593620 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:44.594195 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:44.593685 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:44.594195 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:44.593811 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:44.594299 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:44.594216 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:44.594352 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:44.594302 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:46.214257 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:46.214216 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:46.214751 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.214398 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:46.214751 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:46.214432 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:46.214751 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.214488 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs podName:74858031-b9e7-4b68-83b7-9cd8e9b22824 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:54.21446725 +0000 UTC m=+18.089550688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs") pod "network-metrics-daemon-9z5n8" (UID: "74858031-b9e7-4b68-83b7-9cd8e9b22824") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:46.214751 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.214545 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:46.214751 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.214595 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret podName:5b33b423-4de4-41fa-85de-d5ae343c63aa nodeName:}" failed. No retries permitted until 2026-07-02 14:15:54.214580237 +0000 UTC m=+18.089663675 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret") pod "global-pull-secret-syncer-df6mz" (UID: "5b33b423-4de4-41fa-85de-d5ae343c63aa") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:46.315520 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:46.315482 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhzw\" (UniqueName: \"kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw\") pod \"network-check-target-rzbpn\" (UID: \"b5a88716-1c0d-47f1-95b5-50c0b04d89c1\") " pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:46.315709 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.315664 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:15:46.315709 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.315688 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:15:46.315709 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.315702 2569 projected.go:196] Error preparing data for projected volume kube-api-access-9nhzw for pod openshift-network-diagnostics/network-check-target-rzbpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:46.315875 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.315773 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw podName:b5a88716-1c0d-47f1-95b5-50c0b04d89c1 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:54.315754995 +0000 UTC m=+18.190838448 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9nhzw" (UniqueName: "kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw") pod "network-check-target-rzbpn" (UID: "b5a88716-1c0d-47f1-95b5-50c0b04d89c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:46.593860 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:46.593744 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:46.594020 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.593854 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:46.594202 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:46.594170 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:46.594342 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.594297 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:46.594342 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:46.594330 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:46.594465 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:46.594408 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:48.593502 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:48.593467 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:48.594049 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:48.593601 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:48.594049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:48.593467 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:48.594172 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:48.594061 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:48.594253 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:48.594218 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:48.594350 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:48.594323 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:50.593603 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:50.593514 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:50.594011 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:50.593514 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:50.594011 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:50.593657 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:50.594011 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:50.593652 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:50.594011 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:50.593797 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:50.594011 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:50.593860 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:52.592894 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:52.592861 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:52.592894 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:52.592881 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:52.593421 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:52.592875 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:52.593421 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:52.592987 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:52.593421 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:52.593124 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:52.593421 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:52.593196 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:54.274457 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:54.274411 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:54.274972 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:54.274500 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:54.274972 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.274580 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:54.274972 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.274643 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:54.274972 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.274653 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret podName:5b33b423-4de4-41fa-85de-d5ae343c63aa nodeName:}" failed. No retries permitted until 2026-07-02 14:16:10.274633952 +0000 UTC m=+34.149717408 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret") pod "global-pull-secret-syncer-df6mz" (UID: "5b33b423-4de4-41fa-85de-d5ae343c63aa") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:15:54.274972 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.274716 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs podName:74858031-b9e7-4b68-83b7-9cd8e9b22824 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:10.274697874 +0000 UTC m=+34.149781313 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs") pod "network-metrics-daemon-9z5n8" (UID: "74858031-b9e7-4b68-83b7-9cd8e9b22824") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:15:54.375880 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:54.375840 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhzw\" (UniqueName: \"kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw\") pod \"network-check-target-rzbpn\" (UID: \"b5a88716-1c0d-47f1-95b5-50c0b04d89c1\") " pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:54.376064 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.376042 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:15:54.376119 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.376069 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:15:54.376119 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.376080 2569 projected.go:196] Error preparing data for projected volume kube-api-access-9nhzw for pod openshift-network-diagnostics/network-check-target-rzbpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:54.376209 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.376128 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw podName:b5a88716-1c0d-47f1-95b5-50c0b04d89c1 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:10.376115277 +0000 UTC m=+34.251198710 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9nhzw" (UniqueName: "kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw") pod "network-check-target-rzbpn" (UID: "b5a88716-1c0d-47f1-95b5-50c0b04d89c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:15:54.592912 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:54.592824 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:54.593078 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:54.592824 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:54.593078 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.592945 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:54.593078 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:54.592959 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:54.593246 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.593090 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:54.593246 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:54.593193 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:56.593798 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:56.593756 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:56.594243 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:56.593922 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:56.594243 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:56.593950 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:56.594243 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:56.593990 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:56.594243 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:56.594025 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:56.594243 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:56.594102 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:58.592603 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:58.592584 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:15:58.592951 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:58.592589 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:15:58.592951 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:58.592694 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:15:58.592951 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:58.592679 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:15:58.592951 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:58.592809 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:15:58.592951 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:15:58.592893 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:15:58.776402 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:58.775912 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dcshg" event={"ID":"2833a111-c1fc-4670-92e5-9dfacb75df80","Type":"ContainerStarted","Data":"16b98c23897efd2b0f0f55f3084f94240bb0de828e2fa22dcd2e563db91d361d"} Jul 02 14:15:58.795857 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:58.795499 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" podStartSLOduration=21.795481028 podStartE2EDuration="21.795481028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:15:41.749053834 +0000 UTC m=+5.624137290" watchObservedRunningTime="2026-07-02 14:15:58.795481028 +0000 UTC m=+22.670564485" Jul 02 14:15:59.792957 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.792708 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8jkcr" event={"ID":"8cc01e91-17e6-465e-b77d-acceac9e4a27","Type":"ContainerStarted","Data":"08c93c58980ff54c53f9ecd3188c02574b196c079f1847f05c2059553d44396d"} Jul 02 14:15:59.794160 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.794141 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mf4f7" event={"ID":"ea0e1336-22b5-4c5c-b34d-e4a469f0bc60","Type":"ContainerStarted","Data":"7172b940ad72145a55e0284c16740693f6157aef8c523675dd5c076f2d35c480"} Jul 02 14:15:59.795391 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.795364 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-thntg" event={"ID":"d625f978-90bd-4f61-b0a2-644876bcf84d","Type":"ContainerStarted","Data":"61ded385f79f9395160d86538ece32fb9d9c94f5cfa7df4d704e46d27a56af15"} Jul 02 14:15:59.796725 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.796702 2569 generic.go:356] "Generic (PLEG): container finished" podID="2b739ac8-5c98-4d5a-a3bf-133c98962b46" containerID="33b1f0ab5dadaf4aea72f5c028712b9a3203fb34ff3e1993e3e02cf4422da994" exitCode=0 Jul 02 14:15:59.796806 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.796777 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhddh" event={"ID":"2b739ac8-5c98-4d5a-a3bf-133c98962b46","Type":"ContainerDied","Data":"33b1f0ab5dadaf4aea72f5c028712b9a3203fb34ff3e1993e3e02cf4422da994"} Jul 02 14:15:59.798252 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.798233 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-crjcx" event={"ID":"4948db6d-308b-4367-927b-47c82a9ad85d","Type":"ContainerStarted","Data":"2d70bcbb46673b595c58beea062af844f97116e5810c22a2d5e08c18d5b5f0bc"} Jul 02 14:15:59.799675 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.799639 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" event={"ID":"673457cc-befb-4acf-a4cc-270c97edfc9c","Type":"ContainerStarted","Data":"0ca3cd9f4dfb4b42aa08097c5a10ada7ff926d89d83f05bd3a6ca49d363aa540"} Jul 02 14:15:59.802234 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.802218 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:15:59.802574 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.802549 2569 generic.go:356] "Generic (PLEG): container finished" podID="fb98cb35-0b85-40e4-9a88-54f73548fb3d" containerID="fc4e0a8ed743749949fbdf70002e8758534e60ef9929da66069c3e87d0e0c463" exitCode=1 Jul 02 14:15:59.802648 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.802610 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" event={"ID":"fb98cb35-0b85-40e4-9a88-54f73548fb3d","Type":"ContainerStarted","Data":"4308d2098e9464135c807c27b11c60c85b68367bf743dd5c2390667b84c5ffb9"} Jul 02 14:15:59.802695 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.802679 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" event={"ID":"fb98cb35-0b85-40e4-9a88-54f73548fb3d","Type":"ContainerStarted","Data":"c5c5a469d8a70a6938861c6ef68da515b0d2f9866cb32e93b11274dd930821ae"} Jul 02 14:15:59.802737 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.802726 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" event={"ID":"fb98cb35-0b85-40e4-9a88-54f73548fb3d","Type":"ContainerStarted","Data":"b1bc48ff57812ac7cb4c58334ead1cbb2b6ef6e9824115334ee8351475171ce2"} Jul 02 14:15:59.802777 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.802767 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" event={"ID":"fb98cb35-0b85-40e4-9a88-54f73548fb3d","Type":"ContainerStarted","Data":"61b4f72da36c6b31e5abdb25bee77fbefa11b7b45bda45d59c4130289593e1ac"} Jul 02 14:15:59.802821 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.802809 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" event={"ID":"fb98cb35-0b85-40e4-9a88-54f73548fb3d","Type":"ContainerDied","Data":"fc4e0a8ed743749949fbdf70002e8758534e60ef9929da66069c3e87d0e0c463"} Jul 02 14:15:59.802853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.802846 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" event={"ID":"fb98cb35-0b85-40e4-9a88-54f73548fb3d","Type":"ContainerStarted","Data":"2b4bf76beeccc4261d030b0f1ced573940242e054ab0e8c9d712728c21980fb9"} Jul 02 14:15:59.811125 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.811089 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dcshg" podStartSLOduration=9.741665486 podStartE2EDuration="23.811077354s" totalImagesPullingTime="14.069411868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:15:58.794697873 +0000 UTC m=+22.669781546" watchObservedRunningTime="2026-07-02 14:15:59.811077354 +0000 UTC m=+23.686160808" Jul 02 14:15:59.811215 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.811203 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8jkcr" podStartSLOduration=5.038974637 podStartE2EDuration="23.811199554s" totalImagesPullingTime="18.772224917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:15:59.810457202 +0000 UTC m=+23.685540652" watchObservedRunningTime="2026-07-02 14:15:59.811199554 +0000 UTC m=+23.686283007" Jul 02 14:15:59.811559 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.811545 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:15:59.811975 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.811962 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:15:59.826078 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.826044 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mf4f7" podStartSLOduration=5.02690334 podStartE2EDuration="23.826032754s" totalImagesPullingTime="18.799129414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:15:59.825567098 +0000 UTC m=+23.700650555" watchObservedRunningTime="2026-07-02 14:15:59.826032754 +0000 UTC m=+23.701116208" Jul 02 14:15:59.869442 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.869367 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-crjcx" podStartSLOduration=4.856673186 podStartE2EDuration="23.869353454s" totalImagesPullingTime="19.012680268s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:15:59.869059097 +0000 UTC m=+23.744142552" watchObservedRunningTime="2026-07-02 14:15:59.869353454 +0000 UTC m=+23.744436909" Jul 02 14:15:59.887719 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:15:59.887681 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-thntg" podStartSLOduration=5.086892164 podStartE2EDuration="23.887667892s" totalImagesPullingTime="18.800775728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:15:59.88748809 +0000 UTC m=+23.762571546" watchObservedRunningTime="2026-07-02 14:15:59.887667892 +0000 UTC m=+23.762751347" Jul 02 14:16:00.270804 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.270783 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 14:16:00.593638 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.593558 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:00.593895 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.593557 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:00.593895 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:00.593698 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:16:00.593895 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:00.593769 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:16:00.593895 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.593558 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:00.593895 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:00.593873 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:16:00.711317 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.711203 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T14:16:00.270799364Z","UUID":"a4be7447-5c3f-4171-ad9b-65d71100ff5c","Handler":null,"Name":"","Endpoint":""} Jul 02 14:16:00.713084 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.713051 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 14:16:00.713215 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.713093 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 14:16:00.806606 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.806565 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kr8t2" event={"ID":"1942e8f3-282c-427b-af88-9bf72887e1e6","Type":"ContainerStarted","Data":"af7b3c7c454cb90eafc53f1a1b7fdef0b77f4e49755c20e6b38a75382ff6b2b4"} Jul 02 14:16:00.808663 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.808626 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" event={"ID":"673457cc-befb-4acf-a4cc-270c97edfc9c","Type":"ContainerStarted","Data":"c5e350860cef7f569236f868afe40ee973fc75df85beb15b91df92cc9e13204c"} Jul 02 14:16:00.809483 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.809421 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:16:00.809653 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.809637 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dcshg" Jul 02 14:16:00.822669 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:00.822627 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kr8t2" podStartSLOduration=6.049634048 podStartE2EDuration="24.822612421s" totalImagesPullingTime="18.772978373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:00.822565227 +0000 UTC m=+24.697648682" watchObservedRunningTime="2026-07-02 14:16:00.822612421 +0000 UTC m=+24.697695876" Jul 02 14:16:01.814879 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:01.814670 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:16:01.815347 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:01.815169 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" event={"ID":"fb98cb35-0b85-40e4-9a88-54f73548fb3d","Type":"ContainerStarted","Data":"2f1358e3bb3654fa2b9a3d4ec6c63506c45070be83df2dc6dcac7b2be505ef9d"} Jul 02 14:16:02.593095 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:02.593049 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:02.593095 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:02.593091 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:02.593486 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:02.593066 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:02.593486 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:02.593190 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:16:02.593486 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:02.593261 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:16:02.593486 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:02.593351 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:16:02.819391 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:02.819347 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" event={"ID":"673457cc-befb-4acf-a4cc-270c97edfc9c","Type":"ContainerStarted","Data":"baae52d338a84baaddc9e996e530ce67e30bc00121670fcc22f97a316f3c8eec"} Jul 02 14:16:02.839319 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:02.839262 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmtv7" podStartSLOduration=5.133014936 podStartE2EDuration="26.839246431s" totalImagesPullingTime="21.706231495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:02.837953338 +0000 UTC m=+26.713036794" watchObservedRunningTime="2026-07-02 14:16:02.839246431 +0000 UTC m=+26.714329886" Jul 02 14:16:03.828365 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:03.828017 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:16:03.829644 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:03.829514 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" event={"ID":"fb98cb35-0b85-40e4-9a88-54f73548fb3d","Type":"ContainerStarted","Data":"2f0d4075cd8baa174a8ddb188e6f30e3e162fcb87e5f6a6d1c728a945ea50874"} Jul 02 14:16:03.829644 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:03.829534 2569 scope.go:117] "RemoveContainer" containerID="fc4e0a8ed743749949fbdf70002e8758534e60ef9929da66069c3e87d0e0c463" Jul 02 14:16:03.829885 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:03.829643 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:16:03.829885 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:03.829697 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:16:03.829885 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:03.829744 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:16:03.851077 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:03.851050 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:16:03.852168 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:03.852145 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:16:04.592938 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:04.592753 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:04.593112 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:04.592791 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:04.593112 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:04.593010 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:16:04.593219 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:04.592815 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:04.593219 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:04.593138 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:16:04.593219 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:04.593199 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:16:04.832658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:04.832616 2569 generic.go:356] "Generic (PLEG): container finished" podID="2b739ac8-5c98-4d5a-a3bf-133c98962b46" containerID="1d31f95c61509abe225fd877034b0213348cdaf40eacf0e6f929b95476253f92" exitCode=0 Jul 02 14:16:04.833101 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:04.832698 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhddh" event={"ID":"2b739ac8-5c98-4d5a-a3bf-133c98962b46","Type":"ContainerDied","Data":"1d31f95c61509abe225fd877034b0213348cdaf40eacf0e6f929b95476253f92"} Jul 02 14:16:04.836316 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:04.836299 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:16:04.836712 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:04.836693 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" event={"ID":"fb98cb35-0b85-40e4-9a88-54f73548fb3d","Type":"ContainerStarted","Data":"e7ef971d8b9cb178fc39c86052d49581d2facbfbe0fbdf7b1d35d56f6e1f3b1e"} Jul 02 14:16:04.888870 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:04.888831 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" podStartSLOduration=10.033595944 podStartE2EDuration="28.888812715s" totalImagesPullingTime="18.855216771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:04.887146043 +0000 UTC m=+28.762229500" watchObservedRunningTime="2026-07-02 14:16:04.888812715 +0000 UTC m=+28.763896161" Jul 02 14:16:05.984584 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:05.984554 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rzbpn"] Jul 02 14:16:05.984983 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:05.984682 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:05.984983 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:05.984798 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:16:05.987021 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:05.986991 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9z5n8"] Jul 02 14:16:05.987152 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:05.987118 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:05.987262 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:05.987238 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:16:05.987487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:05.987443 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-df6mz"] Jul 02 14:16:05.987582 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:05.987568 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:05.987689 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:05.987664 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:16:06.843083 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:06.842833 2569 generic.go:356] "Generic (PLEG): container finished" podID="2b739ac8-5c98-4d5a-a3bf-133c98962b46" containerID="278732783dacd842086900fa9241adce9ddf0b335e678bd2dc85b9f173fc533c" exitCode=0 Jul 02 14:16:06.843254 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:06.842912 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhddh" event={"ID":"2b739ac8-5c98-4d5a-a3bf-133c98962b46","Type":"ContainerDied","Data":"278732783dacd842086900fa9241adce9ddf0b335e678bd2dc85b9f173fc533c"} Jul 02 14:16:07.593433 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:07.593399 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:07.593433 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:07.593425 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:07.594060 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:07.593465 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:07.594060 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:07.593557 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:16:07.594060 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:07.593635 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:16:07.594060 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:07.593719 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:16:08.849857 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:08.849826 2569 generic.go:356] "Generic (PLEG): container finished" podID="2b739ac8-5c98-4d5a-a3bf-133c98962b46" containerID="dde85ec767043b6e58f2ff255ca32bc803788945850c5f3a624118ca58481681" exitCode=0 Jul 02 14:16:08.850517 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:08.849863 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhddh" event={"ID":"2b739ac8-5c98-4d5a-a3bf-133c98962b46","Type":"ContainerDied","Data":"dde85ec767043b6e58f2ff255ca32bc803788945850c5f3a624118ca58481681"} Jul 02 14:16:09.593340 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:09.593310 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:09.593510 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:09.593310 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:09.593510 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:09.593407 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzbpn" podUID="b5a88716-1c0d-47f1-95b5-50c0b04d89c1" Jul 02 14:16:09.593510 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:09.593490 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-df6mz" podUID="5b33b423-4de4-41fa-85de-d5ae343c63aa" Jul 02 14:16:09.593510 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:09.593324 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:09.593671 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:09.593581 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z5n8" podUID="74858031-b9e7-4b68-83b7-9cd8e9b22824" Jul 02 14:16:10.298479 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:10.298188 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:10.298479 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:10.298469 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:10.298931 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:10.298339 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:10.298931 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:10.298574 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret podName:5b33b423-4de4-41fa-85de-d5ae343c63aa nodeName:}" failed. No retries permitted until 2026-07-02 14:16:42.298548665 +0000 UTC m=+66.173632103 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret") pod "global-pull-secret-syncer-df6mz" (UID: "5b33b423-4de4-41fa-85de-d5ae343c63aa") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:10.298931 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:10.298589 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:10.298931 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:10.298638 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs podName:74858031-b9e7-4b68-83b7-9cd8e9b22824 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:42.298618828 +0000 UTC m=+66.173702261 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs") pod "network-metrics-daemon-9z5n8" (UID: "74858031-b9e7-4b68-83b7-9cd8e9b22824") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:10.399716 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:10.399683 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhzw\" (UniqueName: \"kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw\") pod \"network-check-target-rzbpn\" (UID: \"b5a88716-1c0d-47f1-95b5-50c0b04d89c1\") " pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:10.399898 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:10.399815 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:16:10.399898 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:10.399838 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:16:10.399898 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:10.399853 2569 projected.go:196] Error preparing data for projected volume kube-api-access-9nhzw for pod openshift-network-diagnostics/network-check-target-rzbpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:10.400010 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:10.399934 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw podName:b5a88716-1c0d-47f1-95b5-50c0b04d89c1 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:42.399916561 +0000 UTC m=+66.274999997 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9nhzw" (UniqueName: "kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw") pod "network-check-target-rzbpn" (UID: "b5a88716-1c0d-47f1-95b5-50c0b04d89c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:11.385418 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.385379 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeReady" Jul 02 14:16:11.385894 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.385549 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 14:16:11.425215 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.425084 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65555b575-h4qjs"] Jul 02 14:16:11.427035 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.427010 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.430835 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.430807 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 14:16:11.431431 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.431391 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 14:16:11.432001 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.431597 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lmkdd\"" Jul 02 14:16:11.432001 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.431865 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 14:16:11.437677 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.437658 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 14:16:11.438642 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.438624 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2bkg2"] Jul 02 14:16:11.440513 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.440484 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.441734 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.441714 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65555b575-h4qjs"] Jul 02 14:16:11.443149 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.443125 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v6rxw\"" Jul 02 14:16:11.443247 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.443230 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 14:16:11.443648 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.443633 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 14:16:11.444706 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.444689 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pkk26"] Jul 02 14:16:11.446459 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.446430 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pkk26" Jul 02 14:16:11.449506 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.449489 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 14:16:11.449602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.449540 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 14:16:11.449602 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.449590 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pc47c\"" Jul 02 14:16:11.449846 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.449830 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 14:16:11.456882 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.456843 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pkk26"] Jul 02 14:16:11.459070 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.459037 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2bkg2"] Jul 02 14:16:11.592696 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.592666 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:11.592696 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.592688 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:11.592932 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.592804 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:11.596134 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.595940 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 14:16:11.596134 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.595951 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s6j8d\"" Jul 02 14:16:11.596134 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.595999 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 14:16:11.596134 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.596020 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 14:16:11.596416 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.596318 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m6dwx\"" Jul 02 14:16:11.596416 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.596357 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 14:16:11.607759 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.607733 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-bound-sa-token\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.607857 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.607768 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc-cert\") pod \"ingress-canary-pkk26\" (UID: \"3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc\") " pod="openshift-ingress-canary/ingress-canary-pkk26" Jul 02 14:16:11.607857 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.607802 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-image-registry-private-configuration\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.607857 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.607827 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-installation-pull-secrets\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.607857 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.607850 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5ba9c566-6099-40fd-9867-2e2dea90f315-config-volume\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.608069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.607901 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-trusted-ca\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.608069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.607935 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba9c566-6099-40fd-9867-2e2dea90f315-tmp-dir\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.608069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.607977 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.608069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.608019 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-ca-trust-extracted\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.608069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.608052 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.608301 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.608145 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np9zg\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-kube-api-access-np9zg\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.608301 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.608168 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqdpv\" (UniqueName: \"kubernetes.io/projected/3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc-kube-api-access-vqdpv\") pod \"ingress-canary-pkk26\" (UID: \"3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc\") " pod="openshift-ingress-canary/ingress-canary-pkk26" Jul 02 14:16:11.608301 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.608187 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-certificates\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.608301 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.608207 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gglnx\" (UniqueName: \"kubernetes.io/projected/5ba9c566-6099-40fd-9867-2e2dea90f315-kube-api-access-gglnx\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.709586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.709490 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba9c566-6099-40fd-9867-2e2dea90f315-tmp-dir\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.709586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.709537 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.709586 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.709561 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-ca-trust-extracted\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.709912 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.709589 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.709912 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.709669 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np9zg\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-kube-api-access-np9zg\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.709912 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:11.709693 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:16:11.709912 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:11.709714 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65555b575-h4qjs: secret "image-registry-tls" not found Jul 02 14:16:11.709912 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.709696 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqdpv\" (UniqueName: \"kubernetes.io/projected/3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc-kube-api-access-vqdpv\") pod \"ingress-canary-pkk26\" (UID: \"3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc\") " pod="openshift-ingress-canary/ingress-canary-pkk26" Jul 02 14:16:11.709912 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:11.709788 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls podName:8dc6dd9c-6110-413a-9ecf-4d414e7e5e65 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:12.209768497 +0000 UTC m=+36.084851944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls") pod "image-registry-65555b575-h4qjs" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65") : secret "image-registry-tls" not found Jul 02 14:16:11.709912 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.709888 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba9c566-6099-40fd-9867-2e2dea90f315-tmp-dir\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.709912 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:11.709900 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:16:11.710270 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:11.709956 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls podName:5ba9c566-6099-40fd-9867-2e2dea90f315 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:12.209941839 +0000 UTC m=+36.085025290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls") pod "dns-default-2bkg2" (UID: "5ba9c566-6099-40fd-9867-2e2dea90f315") : secret "dns-default-metrics-tls" not found Jul 02 14:16:11.710270 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.709889 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-certificates\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.710270 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.709976 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-ca-trust-extracted\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.710270 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.709997 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gglnx\" (UniqueName: \"kubernetes.io/projected/5ba9c566-6099-40fd-9867-2e2dea90f315-kube-api-access-gglnx\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.710270 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.710039 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-bound-sa-token\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.710270 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.710060 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc-cert\") pod \"ingress-canary-pkk26\" (UID: \"3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc\") " pod="openshift-ingress-canary/ingress-canary-pkk26" Jul 02 14:16:11.710270 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.710098 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-image-registry-private-configuration\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.710270 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.710164 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-installation-pull-secrets\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.710270 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.710218 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5ba9c566-6099-40fd-9867-2e2dea90f315-config-volume\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.710705 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.710250 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-trusted-ca\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.710705 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.710398 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-certificates\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.710848 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.710826 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5ba9c566-6099-40fd-9867-2e2dea90f315-config-volume\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.711389 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.711334 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-trusted-ca\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.714655 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.714632 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-installation-pull-secrets\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.714773 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.714727 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-image-registry-private-configuration\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.714850 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.714816 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc-cert\") pod \"ingress-canary-pkk26\" (UID: \"3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc\") " pod="openshift-ingress-canary/ingress-canary-pkk26" Jul 02 14:16:11.721960 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.721895 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gglnx\" (UniqueName: \"kubernetes.io/projected/5ba9c566-6099-40fd-9867-2e2dea90f315-kube-api-access-gglnx\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:11.722078 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.722015 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-bound-sa-token\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.722135 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.722086 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np9zg\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-kube-api-access-np9zg\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:11.722260 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.722206 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqdpv\" (UniqueName: \"kubernetes.io/projected/3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc-kube-api-access-vqdpv\") pod \"ingress-canary-pkk26\" (UID: \"3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc\") " pod="openshift-ingress-canary/ingress-canary-pkk26" Jul 02 14:16:11.759895 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.759864 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pkk26" Jul 02 14:16:11.891592 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:11.891560 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pkk26"] Jul 02 14:16:11.894683 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:11.894657 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ee9e4a2_e775_4a5e_b02b_021c2c31a1dc.slice/crio-9707ed4dc0bc9fea242f5d6f32ce697017f9f829f80813ed66e033a25628980f WatchSource:0}: Error finding container 9707ed4dc0bc9fea242f5d6f32ce697017f9f829f80813ed66e033a25628980f: Status 404 returned error can't find the container with id 9707ed4dc0bc9fea242f5d6f32ce697017f9f829f80813ed66e033a25628980f Jul 02 14:16:12.214440 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:12.214403 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:12.214440 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:12.214457 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:12.214697 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:12.214578 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:16:12.214697 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:12.214584 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:16:12.214697 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:12.214652 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls podName:5ba9c566-6099-40fd-9867-2e2dea90f315 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:13.214632438 +0000 UTC m=+37.089715886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls") pod "dns-default-2bkg2" (UID: "5ba9c566-6099-40fd-9867-2e2dea90f315") : secret "dns-default-metrics-tls" not found Jul 02 14:16:12.214697 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:12.214596 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65555b575-h4qjs: secret "image-registry-tls" not found Jul 02 14:16:12.214895 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:12.214734 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls podName:8dc6dd9c-6110-413a-9ecf-4d414e7e5e65 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:13.214716882 +0000 UTC m=+37.089800315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls") pod "image-registry-65555b575-h4qjs" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65") : secret "image-registry-tls" not found Jul 02 14:16:12.861380 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:12.861339 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pkk26" event={"ID":"3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc","Type":"ContainerStarted","Data":"9707ed4dc0bc9fea242f5d6f32ce697017f9f829f80813ed66e033a25628980f"} Jul 02 14:16:13.222886 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:13.222847 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:13.222886 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:13.222899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:13.223127 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:13.223038 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:16:13.223127 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:13.223061 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65555b575-h4qjs: secret "image-registry-tls" not found Jul 02 14:16:13.223127 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:13.223093 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:16:13.223127 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:13.223124 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls podName:8dc6dd9c-6110-413a-9ecf-4d414e7e5e65 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:15.223103209 +0000 UTC m=+39.098186645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls") pod "image-registry-65555b575-h4qjs" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65") : secret "image-registry-tls" not found Jul 02 14:16:13.223317 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:13.223153 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls podName:5ba9c566-6099-40fd-9867-2e2dea90f315 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:15.223136085 +0000 UTC m=+39.098219540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls") pod "dns-default-2bkg2" (UID: "5ba9c566-6099-40fd-9867-2e2dea90f315") : secret "dns-default-metrics-tls" not found Jul 02 14:16:14.870219 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:14.869965 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhddh" event={"ID":"2b739ac8-5c98-4d5a-a3bf-133c98962b46","Type":"ContainerStarted","Data":"7941f584bba071eee2e4eec4580ebebb80334422e65adbab3ccc41cae7f63278"} Jul 02 14:16:14.871615 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:14.871586 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pkk26" event={"ID":"3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc","Type":"ContainerStarted","Data":"260da009e9b75c1960566889f9090f7944afb276d034da29a99036c9be08909f"} Jul 02 14:16:14.912609 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:14.912549 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pkk26" podStartSLOduration=1.10365289 podStartE2EDuration="3.912528265s" totalImagesPullingTime="2.808875375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:14.911834934 +0000 UTC m=+38.786918389" watchObservedRunningTime="2026-07-02 14:16:14.912528265 +0000 UTC m=+38.787611721" Jul 02 14:16:15.239573 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:15.239538 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:15.239573 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:15.239579 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:15.239767 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:15.239690 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:16:15.239767 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:15.239744 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls podName:5ba9c566-6099-40fd-9867-2e2dea90f315 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:19.239731574 +0000 UTC m=+43.114815007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls") pod "dns-default-2bkg2" (UID: "5ba9c566-6099-40fd-9867-2e2dea90f315") : secret "dns-default-metrics-tls" not found Jul 02 14:16:15.239841 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:15.239694 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:16:15.239841 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:15.239781 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65555b575-h4qjs: secret "image-registry-tls" not found Jul 02 14:16:15.239841 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:15.239836 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls podName:8dc6dd9c-6110-413a-9ecf-4d414e7e5e65 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:19.239823469 +0000 UTC m=+43.114906916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls") pod "image-registry-65555b575-h4qjs" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65") : secret "image-registry-tls" not found Jul 02 14:16:15.876903 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:15.876873 2569 generic.go:356] "Generic (PLEG): container finished" podID="2b739ac8-5c98-4d5a-a3bf-133c98962b46" containerID="7941f584bba071eee2e4eec4580ebebb80334422e65adbab3ccc41cae7f63278" exitCode=0 Jul 02 14:16:15.877346 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:15.876962 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhddh" event={"ID":"2b739ac8-5c98-4d5a-a3bf-133c98962b46","Type":"ContainerDied","Data":"7941f584bba071eee2e4eec4580ebebb80334422e65adbab3ccc41cae7f63278"} Jul 02 14:16:16.882333 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:16.882297 2569 generic.go:356] "Generic (PLEG): container finished" podID="2b739ac8-5c98-4d5a-a3bf-133c98962b46" containerID="fb23ce81640bef6fffc71f4d5561b175416c6a3d1d8c34dd07affb5cdb6ee214" exitCode=0 Jul 02 14:16:16.882921 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:16.882357 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhddh" event={"ID":"2b739ac8-5c98-4d5a-a3bf-133c98962b46","Type":"ContainerDied","Data":"fb23ce81640bef6fffc71f4d5561b175416c6a3d1d8c34dd07affb5cdb6ee214"} Jul 02 14:16:17.887301 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:17.887261 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhddh" event={"ID":"2b739ac8-5c98-4d5a-a3bf-133c98962b46","Type":"ContainerStarted","Data":"2594af7f03618f53249b5bc3c92cae3e0539189c96fc5c197f19f29aec614fb6"} Jul 02 14:16:17.927442 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:17.927397 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zhddh" podStartSLOduration=10.610130645 podStartE2EDuration="41.927382606s" totalImagesPullingTime="31.317251961s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:17.925007982 +0000 UTC m=+41.800091436" watchObservedRunningTime="2026-07-02 14:16:17.927382606 +0000 UTC m=+41.802466060" Jul 02 14:16:18.952486 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:18.952439 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cbz48"] Jul 02 14:16:18.988416 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:18.988381 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cbz48"] Jul 02 14:16:18.988592 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:18.988534 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:18.992253 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:18.992228 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 14:16:18.992394 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:18.992264 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 14:16:18.993814 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:18.993792 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 14:16:18.993913 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:18.993819 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wgq76\"" Jul 02 14:16:18.993913 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:18.993794 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 14:16:19.165731 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.165690 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks2jr\" (UniqueName: \"kubernetes.io/projected/cf882eea-6e9f-40be-8a75-249601c205de-kube-api-access-ks2jr\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.165907 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.165751 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf882eea-6e9f-40be-8a75-249601c205de-data-volume\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.165907 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.165783 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf882eea-6e9f-40be-8a75-249601c205de-crio-socket\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.165907 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.165803 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.165907 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.165876 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.266314 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.266227 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf882eea-6e9f-40be-8a75-249601c205de-data-volume\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.266314 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.266292 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf882eea-6e9f-40be-8a75-249601c205de-crio-socket\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.266509 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.266317 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.266509 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.266357 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.266509 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.266391 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:19.266509 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.266414 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:19.266509 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.266437 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks2jr\" (UniqueName: \"kubernetes.io/projected/cf882eea-6e9f-40be-8a75-249601c205de-kube-api-access-ks2jr\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.266509 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.266476 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:16:19.266763 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.266532 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:16:19.266763 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.266474 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 14:16:19.266763 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.266552 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65555b575-h4qjs: secret "image-registry-tls" not found Jul 02 14:16:19.266763 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.266553 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf882eea-6e9f-40be-8a75-249601c205de-crio-socket\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.266763 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.266545 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:16:19.266763 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.266539 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls podName:cf882eea-6e9f-40be-8a75-249601c205de nodeName:}" failed. No retries permitted until 2026-07-02 14:16:19.766524347 +0000 UTC m=+43.641607781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbz48" (UID: "cf882eea-6e9f-40be-8a75-249601c205de") : secret "insights-runtime-extractor-tls" not found Jul 02 14:16:19.266763 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.266625 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf882eea-6e9f-40be-8a75-249601c205de-data-volume\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.266763 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.266719 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm podName:cf882eea-6e9f-40be-8a75-249601c205de nodeName:}" failed. No retries permitted until 2026-07-02 14:16:19.766699985 +0000 UTC m=+43.641783419 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbz48" (UID: "cf882eea-6e9f-40be-8a75-249601c205de") : configmap "kube-rbac-proxy" not found Jul 02 14:16:19.266763 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.266760 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls podName:8dc6dd9c-6110-413a-9ecf-4d414e7e5e65 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:27.266745343 +0000 UTC m=+51.141828780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls") pod "image-registry-65555b575-h4qjs" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65") : secret "image-registry-tls" not found Jul 02 14:16:19.267074 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.266775 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls podName:5ba9c566-6099-40fd-9867-2e2dea90f315 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:27.266769581 +0000 UTC m=+51.141853013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls") pod "dns-default-2bkg2" (UID: "5ba9c566-6099-40fd-9867-2e2dea90f315") : secret "dns-default-metrics-tls" not found Jul 02 14:16:19.278406 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.278377 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks2jr\" (UniqueName: \"kubernetes.io/projected/cf882eea-6e9f-40be-8a75-249601c205de-kube-api-access-ks2jr\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.770592 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.770554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.770777 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:19.770605 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:19.770777 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.770694 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 14:16:19.770777 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.770707 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:16:19.770777 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.770760 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm podName:cf882eea-6e9f-40be-8a75-249601c205de nodeName:}" failed. No retries permitted until 2026-07-02 14:16:20.770745471 +0000 UTC m=+44.645828904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbz48" (UID: "cf882eea-6e9f-40be-8a75-249601c205de") : configmap "kube-rbac-proxy" not found Jul 02 14:16:19.770777 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:19.770775 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls podName:cf882eea-6e9f-40be-8a75-249601c205de nodeName:}" failed. No retries permitted until 2026-07-02 14:16:20.770767328 +0000 UTC m=+44.645850761 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbz48" (UID: "cf882eea-6e9f-40be-8a75-249601c205de") : secret "insights-runtime-extractor-tls" not found Jul 02 14:16:20.779862 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:20.779823 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:20.780256 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:20.779872 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:20.780256 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:20.779947 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 14:16:20.780256 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:20.779981 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:16:20.780256 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:20.779995 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm podName:cf882eea-6e9f-40be-8a75-249601c205de nodeName:}" failed. No retries permitted until 2026-07-02 14:16:22.779981504 +0000 UTC m=+46.655064937 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbz48" (UID: "cf882eea-6e9f-40be-8a75-249601c205de") : configmap "kube-rbac-proxy" not found Jul 02 14:16:20.780256 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:20.780098 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls podName:cf882eea-6e9f-40be-8a75-249601c205de nodeName:}" failed. No retries permitted until 2026-07-02 14:16:22.780086931 +0000 UTC m=+46.655170365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbz48" (UID: "cf882eea-6e9f-40be-8a75-249601c205de") : secret "insights-runtime-extractor-tls" not found Jul 02 14:16:21.933985 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:21.933954 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mf4f7_ea0e1336-22b5-4c5c-b34d-e4a469f0bc60/dns-node-resolver/0.log" Jul 02 14:16:22.535185 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:22.535160 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8jkcr_8cc01e91-17e6-465e-b77d-acceac9e4a27/node-ca/0.log" Jul 02 14:16:22.795872 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:22.795789 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:22.795872 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:22.795838 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:22.796054 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:22.795935 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:16:22.796054 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:22.795947 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 14:16:22.796054 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:22.796007 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls podName:cf882eea-6e9f-40be-8a75-249601c205de nodeName:}" failed. No retries permitted until 2026-07-02 14:16:26.795988291 +0000 UTC m=+50.671071851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbz48" (UID: "cf882eea-6e9f-40be-8a75-249601c205de") : secret "insights-runtime-extractor-tls" not found Jul 02 14:16:22.796054 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:22.796028 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm podName:cf882eea-6e9f-40be-8a75-249601c205de nodeName:}" failed. No retries permitted until 2026-07-02 14:16:26.796018895 +0000 UTC m=+50.671102358 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbz48" (UID: "cf882eea-6e9f-40be-8a75-249601c205de") : configmap "kube-rbac-proxy" not found Jul 02 14:16:23.339678 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:23.339620 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pkk26_3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc/serve-healthcheck-canary/0.log" Jul 02 14:16:26.825763 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:26.825722 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:26.825763 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:26.825770 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:26.826178 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:26.825872 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:16:26.826178 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:26.825937 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls podName:cf882eea-6e9f-40be-8a75-249601c205de nodeName:}" failed. No retries permitted until 2026-07-02 14:16:34.825918827 +0000 UTC m=+58.701002261 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbz48" (UID: "cf882eea-6e9f-40be-8a75-249601c205de") : secret "insights-runtime-extractor-tls" not found Jul 02 14:16:26.826431 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:26.826414 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf882eea-6e9f-40be-8a75-249601c205de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:27.329354 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:27.329317 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:27.329354 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:27.329351 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:27.329589 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:27.329476 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:16:27.329589 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:27.329494 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65555b575-h4qjs: secret "image-registry-tls" not found Jul 02 14:16:27.329589 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:27.329542 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls podName:8dc6dd9c-6110-413a-9ecf-4d414e7e5e65 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:43.32952803 +0000 UTC m=+67.204611489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls") pod "image-registry-65555b575-h4qjs" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65") : secret "image-registry-tls" not found Jul 02 14:16:27.329931 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:27.329914 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:16:27.329976 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:27.329967 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls podName:5ba9c566-6099-40fd-9867-2e2dea90f315 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:43.329952544 +0000 UTC m=+67.205035994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls") pod "dns-default-2bkg2" (UID: "5ba9c566-6099-40fd-9867-2e2dea90f315") : secret "dns-default-metrics-tls" not found Jul 02 14:16:34.884837 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:34.884788 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:34.887034 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:34.887013 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf882eea-6e9f-40be-8a75-249601c205de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbz48\" (UID: \"cf882eea-6e9f-40be-8a75-249601c205de\") " pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:34.897994 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:34.897974 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cbz48" Jul 02 14:16:35.019928 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:35.019894 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cbz48"] Jul 02 14:16:35.023158 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:35.023133 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf882eea_6e9f_40be_8a75_249601c205de.slice/crio-f6fd201ff0c277b58c01e4458bf2ca4200fdda5d61ce1ca8462f5f0ee7316ac7 WatchSource:0}: Error finding container f6fd201ff0c277b58c01e4458bf2ca4200fdda5d61ce1ca8462f5f0ee7316ac7: Status 404 returned error can't find the container with id f6fd201ff0c277b58c01e4458bf2ca4200fdda5d61ce1ca8462f5f0ee7316ac7 Jul 02 14:16:35.853015 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:35.852988 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74t6q" Jul 02 14:16:35.928183 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:35.928136 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbz48" event={"ID":"cf882eea-6e9f-40be-8a75-249601c205de","Type":"ContainerStarted","Data":"f6fd201ff0c277b58c01e4458bf2ca4200fdda5d61ce1ca8462f5f0ee7316ac7"} Jul 02 14:16:39.939951 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:39.939915 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbz48" event={"ID":"cf882eea-6e9f-40be-8a75-249601c205de","Type":"ContainerStarted","Data":"8a3ea94d37a79500e91efabcd2ef9fd9e600aba096eee2475625109dbec43779"} Jul 02 14:16:39.940502 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:39.939976 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbz48" event={"ID":"cf882eea-6e9f-40be-8a75-249601c205de","Type":"ContainerStarted","Data":"f155d322954ae82c332be03f7c280441a9ba407ec6cf32c86a3e4e11c82c211e"} Jul 02 14:16:42.344098 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.344055 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:42.344570 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.344133 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:42.347112 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.347087 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 14:16:42.347242 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.347087 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 14:16:42.357012 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.356983 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b33b423-4de4-41fa-85de-d5ae343c63aa-original-pull-secret\") pod \"global-pull-secret-syncer-df6mz\" (UID: \"5b33b423-4de4-41fa-85de-d5ae343c63aa\") " pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:42.357012 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.357002 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74858031-b9e7-4b68-83b7-9cd8e9b22824-metrics-certs\") pod \"network-metrics-daemon-9z5n8\" (UID: \"74858031-b9e7-4b68-83b7-9cd8e9b22824\") " pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:42.445423 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.445383 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhzw\" (UniqueName: \"kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw\") pod \"network-check-target-rzbpn\" (UID: \"b5a88716-1c0d-47f1-95b5-50c0b04d89c1\") " pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:42.448579 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.448555 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 14:16:42.458950 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.458921 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 14:16:42.468666 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.468644 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nhzw\" (UniqueName: \"kubernetes.io/projected/b5a88716-1c0d-47f1-95b5-50c0b04d89c1-kube-api-access-9nhzw\") pod \"network-check-target-rzbpn\" (UID: \"b5a88716-1c0d-47f1-95b5-50c0b04d89c1\") " pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:42.508407 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.508387 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m6dwx\"" Jul 02 14:16:42.511724 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.511706 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-df6mz" Jul 02 14:16:42.516395 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.516370 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z5n8" Jul 02 14:16:42.520501 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.520483 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s6j8d\"" Jul 02 14:16:42.528134 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.528115 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:42.657175 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.657148 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-df6mz"] Jul 02 14:16:42.660513 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:42.660488 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b33b423_4de4_41fa_85de_d5ae343c63aa.slice/crio-aac7f0e952cf4aac00003283797d89de91f09e5b9475b7575e3e154a1cdb161d WatchSource:0}: Error finding container aac7f0e952cf4aac00003283797d89de91f09e5b9475b7575e3e154a1cdb161d: Status 404 returned error can't find the container with id aac7f0e952cf4aac00003283797d89de91f09e5b9475b7575e3e154a1cdb161d Jul 02 14:16:42.681184 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.681160 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rzbpn"] Jul 02 14:16:42.683893 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:42.683871 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a88716_1c0d_47f1_95b5_50c0b04d89c1.slice/crio-fd278d45cf286adc6109b1392dd9d2f800237c4db6a4d41e820194b8ec0f6b4e WatchSource:0}: Error finding container fd278d45cf286adc6109b1392dd9d2f800237c4db6a4d41e820194b8ec0f6b4e: Status 404 returned error can't find the container with id fd278d45cf286adc6109b1392dd9d2f800237c4db6a4d41e820194b8ec0f6b4e Jul 02 14:16:42.694969 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.694949 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9z5n8"] Jul 02 14:16:42.697532 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:42.697508 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74858031_b9e7_4b68_83b7_9cd8e9b22824.slice/crio-f9c4573cb41a9ba8fbad604d4603e6b5e7df708b99ed2fdebd94dc103d3b2947 WatchSource:0}: Error finding container f9c4573cb41a9ba8fbad604d4603e6b5e7df708b99ed2fdebd94dc103d3b2947: Status 404 returned error can't find the container with id f9c4573cb41a9ba8fbad604d4603e6b5e7df708b99ed2fdebd94dc103d3b2947 Jul 02 14:16:42.948901 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.948864 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbz48" event={"ID":"cf882eea-6e9f-40be-8a75-249601c205de","Type":"ContainerStarted","Data":"eca411bf7d06ab97402ea53d7a82228404bcf099d156f1bb971b21b909343ec3"} Jul 02 14:16:42.949978 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.949951 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9z5n8" event={"ID":"74858031-b9e7-4b68-83b7-9cd8e9b22824","Type":"ContainerStarted","Data":"f9c4573cb41a9ba8fbad604d4603e6b5e7df708b99ed2fdebd94dc103d3b2947"} Jul 02 14:16:42.950946 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.950921 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-df6mz" event={"ID":"5b33b423-4de4-41fa-85de-d5ae343c63aa","Type":"ContainerStarted","Data":"aac7f0e952cf4aac00003283797d89de91f09e5b9475b7575e3e154a1cdb161d"} Jul 02 14:16:42.951851 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.951832 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rzbpn" event={"ID":"b5a88716-1c0d-47f1-95b5-50c0b04d89c1","Type":"ContainerStarted","Data":"fd278d45cf286adc6109b1392dd9d2f800237c4db6a4d41e820194b8ec0f6b4e"} Jul 02 14:16:42.970884 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:42.970844 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cbz48" podStartSLOduration=17.565891707 podStartE2EDuration="24.970829613s" totalImagesPullingTime="7.404937906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:42.970373748 +0000 UTC m=+66.845457228" watchObservedRunningTime="2026-07-02 14:16:42.970829613 +0000 UTC m=+66.845913064" Jul 02 14:16:43.351890 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.351802 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:43.351890 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.351844 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:43.354764 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.354738 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ba9c566-6099-40fd-9867-2e2dea90f315-metrics-tls\") pod \"dns-default-2bkg2\" (UID: \"5ba9c566-6099-40fd-9867-2e2dea90f315\") " pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:43.354875 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.354843 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") pod \"image-registry-65555b575-h4qjs\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:43.541296 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.541259 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lmkdd\"" Jul 02 14:16:43.548805 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.548776 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:43.556027 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.556001 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v6rxw\"" Jul 02 14:16:43.563557 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.563532 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:43.702292 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.702262 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65555b575-h4qjs"] Jul 02 14:16:43.705238 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:43.705208 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc6dd9c_6110_413a_9ecf_4d414e7e5e65.slice/crio-491a936feaac3b17dafc5d4e849ac19deffbcb6d2cd5acc0e14750ddd3c4a9cf WatchSource:0}: Error finding container 491a936feaac3b17dafc5d4e849ac19deffbcb6d2cd5acc0e14750ddd3c4a9cf: Status 404 returned error can't find the container with id 491a936feaac3b17dafc5d4e849ac19deffbcb6d2cd5acc0e14750ddd3c4a9cf Jul 02 14:16:43.719520 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.719491 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2bkg2"] Jul 02 14:16:43.723562 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:43.723530 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba9c566_6099_40fd_9867_2e2dea90f315.slice/crio-fd233cf04b44bbea7a6b06e0c3e5cd801c76feee2b4cb0cc938c3eb6a4b44348 WatchSource:0}: Error finding container fd233cf04b44bbea7a6b06e0c3e5cd801c76feee2b4cb0cc938c3eb6a4b44348: Status 404 returned error can't find the container with id fd233cf04b44bbea7a6b06e0c3e5cd801c76feee2b4cb0cc938c3eb6a4b44348 Jul 02 14:16:43.959111 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.958262 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65555b575-h4qjs" event={"ID":"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65","Type":"ContainerStarted","Data":"620d79c204b384144773f3907edcdd1b447de239200f0adf9af5c1b0162677e9"} Jul 02 14:16:43.959111 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.958326 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65555b575-h4qjs" event={"ID":"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65","Type":"ContainerStarted","Data":"491a936feaac3b17dafc5d4e849ac19deffbcb6d2cd5acc0e14750ddd3c4a9cf"} Jul 02 14:16:43.959111 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.959042 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:16:43.961424 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.961367 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bkg2" event={"ID":"5ba9c566-6099-40fd-9867-2e2dea90f315","Type":"ContainerStarted","Data":"fd233cf04b44bbea7a6b06e0c3e5cd801c76feee2b4cb0cc938c3eb6a4b44348"} Jul 02 14:16:43.979470 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:43.979335 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65555b575-h4qjs" podStartSLOduration=49.979311376 podStartE2EDuration="49.979311376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:15:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:43.978162329 +0000 UTC m=+67.853245805" watchObservedRunningTime="2026-07-02 14:16:43.979311376 +0000 UTC m=+67.854394811" Jul 02 14:16:44.165872 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.165462 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62"] Jul 02 14:16:44.169108 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.168253 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.180026 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.179913 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 02 14:16:44.180948 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.180776 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 14:16:44.181883 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.181809 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 14:16:44.184624 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.184604 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 14:16:44.196740 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.196673 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62"] Jul 02 14:16:44.251466 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.251320 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65555b575-h4qjs"] Jul 02 14:16:44.258797 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.258631 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a70c67d6-d420-4d0f-8422-24aff4d0b434-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b58457c8c-rtp62\" (UID: \"a70c67d6-d420-4d0f-8422-24aff4d0b434\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.258797 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.258682 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70c67d6-d420-4d0f-8422-24aff4d0b434-tmp\") pod \"klusterlet-addon-workmgr-5b58457c8c-rtp62\" (UID: \"a70c67d6-d420-4d0f-8422-24aff4d0b434\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.258797 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.258712 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhrjz\" (UniqueName: \"kubernetes.io/projected/a70c67d6-d420-4d0f-8422-24aff4d0b434-kube-api-access-lhrjz\") pod \"klusterlet-addon-workmgr-5b58457c8c-rtp62\" (UID: \"a70c67d6-d420-4d0f-8422-24aff4d0b434\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.294686 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.293646 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67b7477fd9-w82tg"] Jul 02 14:16:44.296758 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.296642 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.323365 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.323298 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67b7477fd9-w82tg"] Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359328 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-trusted-ca\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359371 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-registry-certificates\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359404 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-installation-pull-secrets\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359437 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a70c67d6-d420-4d0f-8422-24aff4d0b434-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b58457c8c-rtp62\" (UID: \"a70c67d6-d420-4d0f-8422-24aff4d0b434\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359508 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-image-registry-private-configuration\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359544 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70c67d6-d420-4d0f-8422-24aff4d0b434-tmp\") pod \"klusterlet-addon-workmgr-5b58457c8c-rtp62\" (UID: \"a70c67d6-d420-4d0f-8422-24aff4d0b434\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359567 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhrjz\" (UniqueName: \"kubernetes.io/projected/a70c67d6-d420-4d0f-8422-24aff4d0b434-kube-api-access-lhrjz\") pod \"klusterlet-addon-workmgr-5b58457c8c-rtp62\" (UID: \"a70c67d6-d420-4d0f-8422-24aff4d0b434\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359595 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-bound-sa-token\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359622 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-registry-tls\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359646 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-ca-trust-extracted\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.359682 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grftg\" (UniqueName: \"kubernetes.io/projected/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-kube-api-access-grftg\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.360876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.360064 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70c67d6-d420-4d0f-8422-24aff4d0b434-tmp\") pod \"klusterlet-addon-workmgr-5b58457c8c-rtp62\" (UID: \"a70c67d6-d420-4d0f-8422-24aff4d0b434\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.366239 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.365222 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a70c67d6-d420-4d0f-8422-24aff4d0b434-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b58457c8c-rtp62\" (UID: \"a70c67d6-d420-4d0f-8422-24aff4d0b434\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.375217 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.375169 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhrjz\" (UniqueName: \"kubernetes.io/projected/a70c67d6-d420-4d0f-8422-24aff4d0b434-kube-api-access-lhrjz\") pod \"klusterlet-addon-workmgr-5b58457c8c-rtp62\" (UID: \"a70c67d6-d420-4d0f-8422-24aff4d0b434\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.460950 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.460872 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-bound-sa-token\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.461118 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.460962 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-registry-tls\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.461118 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.460989 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-ca-trust-extracted\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.461118 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.461026 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grftg\" (UniqueName: \"kubernetes.io/projected/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-kube-api-access-grftg\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.461118 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.461051 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-trusted-ca\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.461118 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.461074 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-registry-certificates\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.461118 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.461095 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-installation-pull-secrets\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.461406 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.461164 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-image-registry-private-configuration\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.462090 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.461871 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-ca-trust-extracted\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.462702 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.462673 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-registry-certificates\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.462808 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.462755 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-trusted-ca\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.464865 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.464831 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-installation-pull-secrets\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.465183 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.465134 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-image-registry-private-configuration\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.465565 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.465539 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-registry-tls\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.471241 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.471197 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-bound-sa-token\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.471628 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.471586 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grftg\" (UniqueName: \"kubernetes.io/projected/c763bfc2-3408-4bd8-adc4-6d1ee07d90f1-kube-api-access-grftg\") pod \"image-registry-67b7477fd9-w82tg\" (UID: \"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1\") " pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.484919 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.484899 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:44.609785 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.609402 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.680761 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.680690 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62"] Jul 02 14:16:44.782511 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.782476 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67b7477fd9-w82tg"] Jul 02 14:16:44.788061 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:44.788025 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc763bfc2_3408_4bd8_adc4_6d1ee07d90f1.slice/crio-268049d0729012f33a4fd3d1128c9f0991eff88867257d8270f55e3c1e011628 WatchSource:0}: Error finding container 268049d0729012f33a4fd3d1128c9f0991eff88867257d8270f55e3c1e011628: Status 404 returned error can't find the container with id 268049d0729012f33a4fd3d1128c9f0991eff88867257d8270f55e3c1e011628 Jul 02 14:16:44.970547 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.969613 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9z5n8" event={"ID":"74858031-b9e7-4b68-83b7-9cd8e9b22824","Type":"ContainerStarted","Data":"8b15c4381b004c89d21c0ebc8296ab0f9e827c78e09e3571fc6461ce9ffcb9d9"} Jul 02 14:16:44.970547 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.969690 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9z5n8" event={"ID":"74858031-b9e7-4b68-83b7-9cd8e9b22824","Type":"ContainerStarted","Data":"78f8b9811fa089d9339ca1ad9930c168264ea74e7073bc51cd19d8c186d2460e"} Jul 02 14:16:44.973078 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.973041 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" event={"ID":"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1","Type":"ContainerStarted","Data":"e6945e7ea79520839848dfaf58f8e62f67ea94ad98e371daad2bd08c057be1a1"} Jul 02 14:16:44.973207 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.973100 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" event={"ID":"c763bfc2-3408-4bd8-adc4-6d1ee07d90f1","Type":"ContainerStarted","Data":"268049d0729012f33a4fd3d1128c9f0991eff88867257d8270f55e3c1e011628"} Jul 02 14:16:44.973635 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.973613 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:16:44.978376 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.978168 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" event={"ID":"a70c67d6-d420-4d0f-8422-24aff4d0b434","Type":"ContainerStarted","Data":"5737b2b2f10fbe2a52f414108dcc312c845c5c2fe3218a984c70d344a0e64301"} Jul 02 14:16:44.992257 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:44.991948 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9z5n8" podStartSLOduration=67.215424915 podStartE2EDuration="1m8.991927844s" totalImagesPullingTime="1.776502929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:44.990541786 +0000 UTC m=+68.865625242" watchObservedRunningTime="2026-07-02 14:16:44.991927844 +0000 UTC m=+68.867011303" Jul 02 14:16:46.007288 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:46.007210 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" podStartSLOduration=2.007186564 podStartE2EDuration="2.007186564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:45.015490008 +0000 UTC m=+68.890573464" watchObservedRunningTime="2026-07-02 14:16:46.007186564 +0000 UTC m=+69.882270020" Jul 02 14:16:48.150220 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:48.150179 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj"] Jul 02 14:16:48.153525 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:48.153503 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" Jul 02 14:16:48.156767 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:48.156746 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 02 14:16:48.157091 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:48.157067 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jnwgt\"" Jul 02 14:16:48.162612 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:48.162590 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj"] Jul 02 14:16:48.191631 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:48.191598 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96a562da-b1d3-4ec9-b253-3d7e69b4e609-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tdkxj\" (UID: \"96a562da-b1d3-4ec9-b253-3d7e69b4e609\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" Jul 02 14:16:48.292197 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:48.292158 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96a562da-b1d3-4ec9-b253-3d7e69b4e609-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tdkxj\" (UID: \"96a562da-b1d3-4ec9-b253-3d7e69b4e609\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" Jul 02 14:16:48.292357 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:48.292306 2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 02 14:16:48.292417 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:48.292375 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96a562da-b1d3-4ec9-b253-3d7e69b4e609-tls-certificates podName:96a562da-b1d3-4ec9-b253-3d7e69b4e609 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:48.792353711 +0000 UTC m=+72.667437165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/96a562da-b1d3-4ec9-b253-3d7e69b4e609-tls-certificates") pod "prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" (UID: "96a562da-b1d3-4ec9-b253-3d7e69b4e609") : secret "prometheus-operator-admission-webhook-tls" not found Jul 02 14:16:48.796734 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:48.796685 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96a562da-b1d3-4ec9-b253-3d7e69b4e609-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tdkxj\" (UID: \"96a562da-b1d3-4ec9-b253-3d7e69b4e609\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" Jul 02 14:16:48.799491 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:48.799463 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96a562da-b1d3-4ec9-b253-3d7e69b4e609-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tdkxj\" (UID: \"96a562da-b1d3-4ec9-b253-3d7e69b4e609\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" Jul 02 14:16:49.064882 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:49.064860 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" Jul 02 14:16:49.250887 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:49.250859 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj"] Jul 02 14:16:49.255389 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:49.255322 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a562da_b1d3_4ec9_b253_3d7e69b4e609.slice/crio-a386e95dd76cce6eaccb66f0b8bb33a319ec851db88a7a7964138562625b9306 WatchSource:0}: Error finding container a386e95dd76cce6eaccb66f0b8bb33a319ec851db88a7a7964138562625b9306: Status 404 returned error can't find the container with id a386e95dd76cce6eaccb66f0b8bb33a319ec851db88a7a7964138562625b9306 Jul 02 14:16:49.998186 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:49.998146 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rzbpn" event={"ID":"b5a88716-1c0d-47f1-95b5-50c0b04d89c1","Type":"ContainerStarted","Data":"a904f82a2fb5c2363bc2ecc35b5886b0850d00edc813bfd65e9e5ea2d750ae03"} Jul 02 14:16:49.998408 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:49.998230 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:16:49.999656 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:49.999627 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" event={"ID":"96a562da-b1d3-4ec9-b253-3d7e69b4e609","Type":"ContainerStarted","Data":"a386e95dd76cce6eaccb66f0b8bb33a319ec851db88a7a7964138562625b9306"} Jul 02 14:16:50.001119 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:50.001087 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-df6mz" event={"ID":"5b33b423-4de4-41fa-85de-d5ae343c63aa","Type":"ContainerStarted","Data":"cf4b7d7caa446bc7944db7485ff68844ea74fdd98bcd7613510f3aa9aba22527"} Jul 02 14:16:50.003094 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:50.003069 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bkg2" event={"ID":"5ba9c566-6099-40fd-9867-2e2dea90f315","Type":"ContainerStarted","Data":"f11d449a54f4bcaa84649b7234f9871c921ddf5fe6997898f694589524e7070b"} Jul 02 14:16:50.003211 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:50.003114 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bkg2" event={"ID":"5ba9c566-6099-40fd-9867-2e2dea90f315","Type":"ContainerStarted","Data":"ed20e646b86dc9165e99df347c8be4ffbeb1317e2faced60be080f429ff3dda9"} Jul 02 14:16:50.003211 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:50.003191 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2bkg2" Jul 02 14:16:50.052487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:50.052402 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rzbpn" podStartSLOduration=67.724923143 podStartE2EDuration="1m14.052380139s" totalImagesPullingTime="6.327456996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:50.023192451 +0000 UTC m=+73.898275908" watchObservedRunningTime="2026-07-02 14:16:50.052380139 +0000 UTC m=+73.927463595" Jul 02 14:16:50.053682 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:50.053636 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2bkg2" podStartSLOduration=33.76702218 podStartE2EDuration="39.053624302s" totalImagesPullingTime="5.286602122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:50.052709339 +0000 UTC m=+73.927792819" watchObservedRunningTime="2026-07-02 14:16:50.053624302 +0000 UTC m=+73.928707759" Jul 02 14:16:50.075547 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:50.075493 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-df6mz" podStartSLOduration=67.725465105 podStartE2EDuration="1m14.075473611s" totalImagesPullingTime="6.350008506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:50.074623462 +0000 UTC m=+73.949706932" watchObservedRunningTime="2026-07-02 14:16:50.075473611 +0000 UTC m=+73.950557063" Jul 02 14:16:52.011702 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.011659 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" event={"ID":"a70c67d6-d420-4d0f-8422-24aff4d0b434","Type":"ContainerStarted","Data":"5b56ec63eb9ef4a1822340bce4a8d8044f93ccdb51bafab525633baee62d2778"} Jul 02 14:16:52.012146 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.011734 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:52.013270 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.013223 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" Jul 02 14:16:52.014117 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.014089 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" event={"ID":"96a562da-b1d3-4ec9-b253-3d7e69b4e609","Type":"ContainerStarted","Data":"c1709951e272e6a2553d15d4ab86cf0aea37089bfdc523ee8267137ef522a1ee"} Jul 02 14:16:52.014254 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.014235 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" Jul 02 14:16:52.019155 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.019134 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" Jul 02 14:16:52.032200 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.032156 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b58457c8c-rtp62" podStartSLOduration=0.813328155 podStartE2EDuration="8.032142149s" totalImagesPullingTime="7.218813994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:52.03104803 +0000 UTC m=+75.906131484" watchObservedRunningTime="2026-07-02 14:16:52.032142149 +0000 UTC m=+75.907225604" Jul 02 14:16:52.049983 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.049941 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tdkxj" podStartSLOduration=1.402964163 podStartE2EDuration="4.049928317s" totalImagesPullingTime="2.646964154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:52.049859593 +0000 UTC m=+75.924943048" watchObservedRunningTime="2026-07-02 14:16:52.049928317 +0000 UTC m=+75.925011772" Jul 02 14:16:52.227579 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.227495 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-94sdj"] Jul 02 14:16:52.229909 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.229894 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.233346 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.233326 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 02 14:16:52.233884 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.233854 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 14:16:52.233970 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.233911 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 14:16:52.233970 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.233940 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lqbbl\"" Jul 02 14:16:52.234309 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.234295 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 14:16:52.235455 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.235429 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 02 14:16:52.243730 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.243710 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-94sdj"] Jul 02 14:16:52.325169 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.325128 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.325358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.325196 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v77hn\" (UniqueName: \"kubernetes.io/projected/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-kube-api-access-v77hn\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.325358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.325238 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.325358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.325266 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.426226 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.426186 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.426226 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.426229 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.426504 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.426265 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.426504 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.426298 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v77hn\" (UniqueName: \"kubernetes.io/projected/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-kube-api-access-v77hn\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.426981 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.426954 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.428790 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.428771 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.428790 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.428785 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.435428 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.435405 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v77hn\" (UniqueName: \"kubernetes.io/projected/1ffc614e-9b60-45ce-aeef-9cf06cadc6ad-kube-api-access-v77hn\") pod \"prometheus-operator-7b7d896497-94sdj\" (UID: \"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.539008 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.538925 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" Jul 02 14:16:52.685351 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:52.685321 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-94sdj"] Jul 02 14:16:52.688290 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:52.688257 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffc614e_9b60_45ce_aeef_9cf06cadc6ad.slice/crio-b590b73f10ab83e0ea38806d093de2bedc83f0006373674eca72ff273f54aebd WatchSource:0}: Error finding container b590b73f10ab83e0ea38806d093de2bedc83f0006373674eca72ff273f54aebd: Status 404 returned error can't find the container with id b590b73f10ab83e0ea38806d093de2bedc83f0006373674eca72ff273f54aebd Jul 02 14:16:53.018469 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:53.018411 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" event={"ID":"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad","Type":"ContainerStarted","Data":"b590b73f10ab83e0ea38806d093de2bedc83f0006373674eca72ff273f54aebd"} Jul 02 14:16:55.027690 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:55.027655 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" event={"ID":"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad","Type":"ContainerStarted","Data":"7de8168c030faf039d01220aee9f5e85a5939690907982effb1b8c3a51cec0f3"} Jul 02 14:16:55.028133 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:55.027714 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" event={"ID":"1ffc614e-9b60-45ce-aeef-9cf06cadc6ad","Type":"ContainerStarted","Data":"f77ba71437e1ae218bfce4abc9519b9d37c4dbb071481c319de0a9b680047d75"} Jul 02 14:16:55.047068 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:55.047020 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-94sdj" podStartSLOduration=1.23852919 podStartE2EDuration="3.047004352s" totalImagesPullingTime="1.808475162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:55.045846088 +0000 UTC m=+78.920929544" watchObservedRunningTime="2026-07-02 14:16:55.047004352 +0000 UTC m=+78.922087807" Jul 02 14:16:56.905562 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.905526 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-q8f6r"] Jul 02 14:16:56.907709 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.907690 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:56.912544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.912524 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 02 14:16:56.912883 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.912853 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 02 14:16:56.914036 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.913925 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-2zmv5\"" Jul 02 14:16:56.914036 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.913998 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 02 14:16:56.938936 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.938909 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-q8f6r"] Jul 02 14:16:56.959479 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.959428 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:56.959608 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.959498 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:56.959608 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.959556 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm9c8\" (UniqueName: \"kubernetes.io/projected/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-api-access-dm9c8\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:56.959728 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.959652 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-metrics-client-ca\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:56.959728 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.959698 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:56.959820 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:56.959736 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.052926 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.052898 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5fqzk"] Jul 02 14:16:57.055827 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.055811 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.060039 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060016 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.060174 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060046 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.060174 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060078 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.060174 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060096 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.060174 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:57.060164 2569 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 02 14:16:57.060368 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:57.060226 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-tls podName:8e9dccc2-45ba-4786-9319-e5bd22be8aaf nodeName:}" failed. No retries permitted until 2026-07-02 14:16:57.560207047 +0000 UTC m=+81.435290480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-tls") pod "kube-state-metrics-845b954455-q8f6r" (UID: "8e9dccc2-45ba-4786-9319-e5bd22be8aaf") : secret "kube-state-metrics-tls" not found Jul 02 14:16:57.060368 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060276 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm9c8\" (UniqueName: \"kubernetes.io/projected/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-api-access-dm9c8\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.060368 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060345 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-metrics-client-ca\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.060552 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060520 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.060786 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060739 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 14:16:57.060786 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060766 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 14:16:57.060786 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060778 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xv2pg\"" Jul 02 14:16:57.060984 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060964 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 14:16:57.060984 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060978 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-metrics-client-ca\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.061090 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.060962 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.063035 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.063015 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.089844 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.089812 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm9c8\" (UniqueName: \"kubernetes.io/projected/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-api-access-dm9c8\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.161354 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.161263 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-wtmp\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.161354 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.161303 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-textfile\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.161354 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.161325 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc9d7792-4953-49cf-9172-f1e7fde3b839-metrics-client-ca\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.161354 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.161350 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-accelerators-collector-config\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.161680 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.161370 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59nfj\" (UniqueName: \"kubernetes.io/projected/dc9d7792-4953-49cf-9172-f1e7fde3b839-kube-api-access-59nfj\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.161680 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.161437 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-tls\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.161680 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.161502 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc9d7792-4953-49cf-9172-f1e7fde3b839-sys\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.161680 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.161524 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.161680 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.161607 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc9d7792-4953-49cf-9172-f1e7fde3b839-root\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.262689 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc9d7792-4953-49cf-9172-f1e7fde3b839-sys\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.262689 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262692 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.262855 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262737 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc9d7792-4953-49cf-9172-f1e7fde3b839-root\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.262855 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262770 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc9d7792-4953-49cf-9172-f1e7fde3b839-sys\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.262855 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262807 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc9d7792-4953-49cf-9172-f1e7fde3b839-root\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.262855 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262775 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-wtmp\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.262855 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262852 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-textfile\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.263014 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262880 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc9d7792-4953-49cf-9172-f1e7fde3b839-metrics-client-ca\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.263014 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262909 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-wtmp\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.263014 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262924 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-accelerators-collector-config\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.263014 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262954 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59nfj\" (UniqueName: \"kubernetes.io/projected/dc9d7792-4953-49cf-9172-f1e7fde3b839-kube-api-access-59nfj\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.263014 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.262992 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-tls\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.263237 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.263202 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-textfile\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.263511 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.263491 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc9d7792-4953-49cf-9172-f1e7fde3b839-metrics-client-ca\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.263598 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.263544 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-accelerators-collector-config\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.265030 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.265013 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.265144 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.265127 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc9d7792-4953-49cf-9172-f1e7fde3b839-node-exporter-tls\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.271912 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.271890 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59nfj\" (UniqueName: \"kubernetes.io/projected/dc9d7792-4953-49cf-9172-f1e7fde3b839-kube-api-access-59nfj\") pod \"node-exporter-5fqzk\" (UID: \"dc9d7792-4953-49cf-9172-f1e7fde3b839\") " pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.369930 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.369897 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5fqzk" Jul 02 14:16:57.380076 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:57.380047 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc9d7792_4953_49cf_9172_f1e7fde3b839.slice/crio-48d66390aec6d72581c27f6b3b52069ee06ec9803d51e1009e00eecc2112ff6e WatchSource:0}: Error finding container 48d66390aec6d72581c27f6b3b52069ee06ec9803d51e1009e00eecc2112ff6e: Status 404 returned error can't find the container with id 48d66390aec6d72581c27f6b3b52069ee06ec9803d51e1009e00eecc2112ff6e Jul 02 14:16:57.566648 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.566553 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.568884 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.568846 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e9dccc2-45ba-4786-9319-e5bd22be8aaf-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-q8f6r\" (UID: \"8e9dccc2-45ba-4786-9319-e5bd22be8aaf\") " pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.816806 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.816725 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" Jul 02 14:16:57.892535 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.892506 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 14:16:57.897151 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.897124 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.901241 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.901203 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 14:16:57.901359 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.901287 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 14:16:57.901633 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.901437 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pj5q4\"" Jul 02 14:16:57.901633 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.901472 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 14:16:57.901824 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.901666 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 14:16:57.901824 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.901739 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 14:16:57.901960 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.901832 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 14:16:57.901960 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.901944 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 14:16:57.901960 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.901956 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 14:16:57.902117 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.902073 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 14:16:57.923223 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.923193 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 14:16:57.957236 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.957215 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-q8f6r"] Jul 02 14:16:57.959655 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:57.959631 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9dccc2_45ba_4786_9319_e5bd22be8aaf.slice/crio-aae91e4fcc598f5371e83bbbf763f9aa0323613bdbd8f7aa72cc7fb94fda3a16 WatchSource:0}: Error finding container aae91e4fcc598f5371e83bbbf763f9aa0323613bdbd8f7aa72cc7fb94fda3a16: Status 404 returned error can't find the container with id aae91e4fcc598f5371e83bbbf763f9aa0323613bdbd8f7aa72cc7fb94fda3a16 Jul 02 14:16:57.968657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968633 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.968751 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968670 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.968751 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968698 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-out\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.968830 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968770 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.968830 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968808 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-web-config\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.968830 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968827 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-volume\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.968973 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968852 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.968973 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968888 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.968973 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968915 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.968973 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968955 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk79m\" (UniqueName: \"kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-kube-api-access-nk79m\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.969096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.968997 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.969096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.969038 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:57.969096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:57.969057 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.041904 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.041870 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fqzk" event={"ID":"dc9d7792-4953-49cf-9172-f1e7fde3b839","Type":"ContainerStarted","Data":"48d66390aec6d72581c27f6b3b52069ee06ec9803d51e1009e00eecc2112ff6e"} Jul 02 14:16:58.043044 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.043017 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" event={"ID":"8e9dccc2-45ba-4786-9319-e5bd22be8aaf","Type":"ContainerStarted","Data":"aae91e4fcc598f5371e83bbbf763f9aa0323613bdbd8f7aa72cc7fb94fda3a16"} Jul 02 14:16:58.069883 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.069808 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.069883 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.069847 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-web-config\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070074 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.069924 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-volume\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070074 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.069984 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070183 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.070100 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070183 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.070137 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070278 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.070181 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk79m\" (UniqueName: \"kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-kube-api-access-nk79m\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070278 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.070214 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070387 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.070273 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070387 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.070308 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070387 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.070334 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070387 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.070370 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.070593 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.070404 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-out\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.072139 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:16:58.072113 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-trusted-ca-bundle podName:92c46c7b-c750-4501-a45f-b3e160dcffe5 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:58.572091684 +0000 UTC m=+82.447175125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5") : configmap references non-existent config key: ca-bundle.crt Jul 02 14:16:58.072785 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.072758 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.073545 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.073197 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.073545 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.073352 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.073545 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.073493 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-volume\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.073766 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.073750 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.073843 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.073824 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.073915 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.073885 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-out\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.074517 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.074301 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-web-config\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.075344 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.075321 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.075736 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.075713 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.075843 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.075829 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.082734 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.082717 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk79m\" (UniqueName: \"kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-kube-api-access-nk79m\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.575039 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.575001 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.576053 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.576022 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.808606 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.808481 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:16:58.886051 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.886018 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-695698fdf9-9jh5p"] Jul 02 14:16:58.893894 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.893863 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:58.897402 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.896894 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 14:16:58.897402 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.896996 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 14:16:58.897402 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.897099 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 14:16:58.897402 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.896894 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-md9k9\"" Jul 02 14:16:58.897402 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.897308 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 14:16:58.898178 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.898156 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 14:16:58.898826 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.898409 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 14:16:58.899105 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.898973 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 14:16:58.904234 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.904212 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695698fdf9-9jh5p"] Jul 02 14:16:58.963720 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.963692 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 14:16:58.979430 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.979403 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-oauth-serving-cert\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:58.979562 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.979471 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-console-config\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:58.979562 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.979525 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-oauth-config\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:58.979562 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.979548 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-serving-cert\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:58.979690 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.979587 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-service-ca\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:58.979690 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:58.979644 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcg6w\" (UniqueName: \"kubernetes.io/projected/199d3194-744c-4470-b236-0afcbc4dfbee-kube-api-access-jcg6w\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.080461 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.080413 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-oauth-serving-cert\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.080635 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.080494 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-console-config\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.080635 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.080539 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-oauth-config\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.080635 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.080562 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-serving-cert\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.080635 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.080588 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-service-ca\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.080839 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.080774 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcg6w\" (UniqueName: \"kubernetes.io/projected/199d3194-744c-4470-b236-0afcbc4dfbee-kube-api-access-jcg6w\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.081124 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.081101 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-oauth-serving-cert\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.081673 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.081651 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-console-config\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.081757 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.081743 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-service-ca\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.083148 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.083106 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-oauth-config\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.083687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.083663 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-serving-cert\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.089966 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.089929 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcg6w\" (UniqueName: \"kubernetes.io/projected/199d3194-744c-4470-b236-0afcbc4dfbee-kube-api-access-jcg6w\") pod \"console-695698fdf9-9jh5p\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.150251 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:59.150212 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c46c7b_c750_4501_a45f_b3e160dcffe5.slice/crio-9f2d0fc712d34e40f7d1eaf0233c7cce4c029f936f0265161fb46dbec4819768 WatchSource:0}: Error finding container 9f2d0fc712d34e40f7d1eaf0233c7cce4c029f936f0265161fb46dbec4819768: Status 404 returned error can't find the container with id 9f2d0fc712d34e40f7d1eaf0233c7cce4c029f936f0265161fb46dbec4819768 Jul 02 14:16:59.205991 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.205963 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:16:59.372217 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:16:59.372189 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695698fdf9-9jh5p"] Jul 02 14:16:59.398308 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:16:59.398275 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod199d3194_744c_4470_b236_0afcbc4dfbee.slice/crio-22479b32839446e4e4ae0eb149dc96dde5f3c18279822ff841d15ca28b783f64 WatchSource:0}: Error finding container 22479b32839446e4e4ae0eb149dc96dde5f3c18279822ff841d15ca28b783f64: Status 404 returned error can't find the container with id 22479b32839446e4e4ae0eb149dc96dde5f3c18279822ff841d15ca28b783f64 Jul 02 14:17:00.009842 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:00.009345 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2bkg2" Jul 02 14:17:00.067220 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:00.065785 2569 generic.go:356] "Generic (PLEG): container finished" podID="dc9d7792-4953-49cf-9172-f1e7fde3b839" containerID="b7e476e2f821dc68a3335c0b3db281ca5f0be4ab7d1948cf72a5f5d198ebb23a" exitCode=0 Jul 02 14:17:00.067220 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:00.065911 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fqzk" event={"ID":"dc9d7792-4953-49cf-9172-f1e7fde3b839","Type":"ContainerDied","Data":"b7e476e2f821dc68a3335c0b3db281ca5f0be4ab7d1948cf72a5f5d198ebb23a"} Jul 02 14:17:00.071424 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:00.069157 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695698fdf9-9jh5p" event={"ID":"199d3194-744c-4470-b236-0afcbc4dfbee","Type":"ContainerStarted","Data":"22479b32839446e4e4ae0eb149dc96dde5f3c18279822ff841d15ca28b783f64"} Jul 02 14:17:00.084889 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:00.084722 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" event={"ID":"8e9dccc2-45ba-4786-9319-e5bd22be8aaf","Type":"ContainerStarted","Data":"a2c2cca01bfb67d43f84495132851d88632206ea0de6265c9615a2e571564623"} Jul 02 14:17:00.084889 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:00.084794 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" event={"ID":"8e9dccc2-45ba-4786-9319-e5bd22be8aaf","Type":"ContainerStarted","Data":"2af326df3dc62f67f327edada0e34f6ae7548e1e56549b3288a15db6f37e91f2"} Jul 02 14:17:00.090089 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:00.089211 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerStarted","Data":"9f2d0fc712d34e40f7d1eaf0233c7cce4c029f936f0265161fb46dbec4819768"} Jul 02 14:17:01.095142 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:01.095109 2569 generic.go:356] "Generic (PLEG): container finished" podID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerID="dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d" exitCode=0 Jul 02 14:17:01.095642 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:01.095289 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerDied","Data":"dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d"} Jul 02 14:17:01.097746 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:01.097712 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fqzk" event={"ID":"dc9d7792-4953-49cf-9172-f1e7fde3b839","Type":"ContainerStarted","Data":"46c1cd2acc23a9060ad1860727e3a37b22235717c57800f9969301a09a02b713"} Jul 02 14:17:01.097838 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:01.097779 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fqzk" event={"ID":"dc9d7792-4953-49cf-9172-f1e7fde3b839","Type":"ContainerStarted","Data":"c3a8945b25961e0c4f7c3f472d6436239ea0cf18f6d32bf36e848ef5b8a2264d"} Jul 02 14:17:01.100320 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:01.100290 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" event={"ID":"8e9dccc2-45ba-4786-9319-e5bd22be8aaf","Type":"ContainerStarted","Data":"33ca5ffa0b886568ff774df74755cd3f9ac9f8fd768648e00f237eb0ed439cea"} Jul 02 14:17:01.248860 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:01.248736 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-q8f6r" podStartSLOduration=3.304534771 podStartE2EDuration="5.248712958s" totalImagesPullingTime="1.944178187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:01.166117286 +0000 UTC m=+85.041200737" watchObservedRunningTime="2026-07-02 14:17:01.248712958 +0000 UTC m=+85.123796413" Jul 02 14:17:02.955566 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.955502 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5fqzk" podStartSLOduration=5.067457223 podStartE2EDuration="6.955478961s" totalImagesPullingTime="1.814521498s" totalInitContainerRuntime="73.50024ms" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:01.249075044 +0000 UTC m=+85.124158499" watchObservedRunningTime="2026-07-02 14:17:02.955478961 +0000 UTC m=+86.830562416" Jul 02 14:17:02.956752 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.956730 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 14:17:02.961362 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.961331 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:02.966501 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.964723 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 14:17:02.966501 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.965044 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 14:17:02.970677 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.970653 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 14:17:02.970862 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.970847 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 14:17:02.971113 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.971094 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 14:17:02.971194 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.971175 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 14:17:02.971304 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.971289 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 14:17:02.971363 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.971349 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 14:17:02.971673 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.971658 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-d7xbv\"" Jul 02 14:17:02.971732 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.971713 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 14:17:02.973069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.972323 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 14:17:02.973069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.972394 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 14:17:02.973069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.972512 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 14:17:02.973069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.972722 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2fdt34uv0jdvu\"" Jul 02 14:17:02.973069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.972831 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 14:17:02.973069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.972856 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 14:17:02.973069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.973045 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 14:17:02.983783 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:02.983752 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 14:17:03.111610 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.111571 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695698fdf9-9jh5p" event={"ID":"199d3194-744c-4470-b236-0afcbc4dfbee","Type":"ContainerStarted","Data":"14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e"} Jul 02 14:17:03.121321 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121291 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121524 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121332 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shhx7\" (UniqueName: \"kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-kube-api-access-shhx7\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121524 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121359 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config-out\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121524 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121391 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121524 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121437 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121524 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121516 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121749 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121550 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121749 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121572 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121749 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121601 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121749 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121627 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121749 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121661 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121742 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121773 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121794 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121814 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121841 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-web-config\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121878 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.121918 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121896 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.122117 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121921 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.122117 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.121941 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.223602 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.223710 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.223756 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.223788 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.223812 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.223838 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.223889 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.223924 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.223967 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.223999 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224041 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224069 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224135 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-web-config\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224173 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224197 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224220 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.224765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224269 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.225937 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224307 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.225937 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224331 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shhx7\" (UniqueName: \"kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-kube-api-access-shhx7\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.225937 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224360 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config-out\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.225937 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.224546 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.225937 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.225262 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.226222 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.226191 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.226640 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.226481 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.227655 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.227302 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.228688 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.228597 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.229714 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.229656 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.230405 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.229979 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config-out\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.230405 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.230367 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-web-config\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.231275 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.231148 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.231275 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.231225 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.232225 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.231633 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.232225 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.231913 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.232225 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.232159 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.232583 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.232559 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.233494 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.233414 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.233584 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.233517 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.233584 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.233523 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.234964 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.234944 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.240359 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.240337 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shhx7\" (UniqueName: \"kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-kube-api-access-shhx7\") pod \"prometheus-k8s-0\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.283869 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.283824 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:03.453649 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.453477 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-695698fdf9-9jh5p" podStartSLOduration=2.389895161 podStartE2EDuration="5.453437942s" totalImagesPullingTime="3.063542781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:03.132930333 +0000 UTC m=+87.008013789" watchObservedRunningTime="2026-07-02 14:17:03.453437942 +0000 UTC m=+87.328521398" Jul 02 14:17:03.453871 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:03.453853 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 14:17:04.117695 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:04.117650 2569 generic.go:356] "Generic (PLEG): container finished" podID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerID="7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d" exitCode=0 Jul 02 14:17:04.118056 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:04.117765 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerDied","Data":"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d"} Jul 02 14:17:04.118056 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:04.117842 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerStarted","Data":"f490da13a56122b0e904eed84ea09424cf20619a90994a4a67001e1dce8d4a23"} Jul 02 14:17:04.613816 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:04.613723 2569 patch_prober.go:28] interesting pod/image-registry-67b7477fd9-w82tg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 14:17:04.613816 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:04.613799 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" podUID="c763bfc2-3408-4bd8-adc4-6d1ee07d90f1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:17:05.124651 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:05.124620 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerStarted","Data":"40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef"} Jul 02 14:17:05.125042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:05.124687 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerStarted","Data":"36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb"} Jul 02 14:17:05.125042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:05.124712 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerStarted","Data":"1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575"} Jul 02 14:17:05.125042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:05.124737 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerStarted","Data":"43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f"} Jul 02 14:17:05.125042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:05.124759 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerStarted","Data":"a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f"} Jul 02 14:17:05.987432 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:05.987358 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:17:06.131275 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:06.131236 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerStarted","Data":"f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b"} Jul 02 14:17:06.168213 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:06.168159 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.114784105 podStartE2EDuration="9.168143989s" totalImagesPullingTime="6.034084109s" totalInitContainerRuntime="19.275775ms" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:06.166086943 +0000 UTC m=+90.041170398" watchObservedRunningTime="2026-07-02 14:17:06.168143989 +0000 UTC m=+90.043227443" Jul 02 14:17:06.992110 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:06.992078 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67b7477fd9-w82tg" Jul 02 14:17:09.206812 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:09.206773 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:17:09.207258 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:09.206883 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:17:09.213226 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:09.213202 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:17:10.155207 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:10.155175 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerStarted","Data":"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe"} Jul 02 14:17:10.159727 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:10.159701 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:17:11.008084 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.008036 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65555b575-h4qjs" podUID="8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" containerName="registry" containerID="cri-o://620d79c204b384144773f3907edcdd1b447de239200f0adf9af5c1b0162677e9" gracePeriod=30 Jul 02 14:17:11.162466 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.162413 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerStarted","Data":"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63"} Jul 02 14:17:11.164066 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.164041 2569 generic.go:356] "Generic (PLEG): container finished" podID="8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" containerID="620d79c204b384144773f3907edcdd1b447de239200f0adf9af5c1b0162677e9" exitCode=0 Jul 02 14:17:11.164201 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.164100 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65555b575-h4qjs" event={"ID":"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65","Type":"ContainerDied","Data":"620d79c204b384144773f3907edcdd1b447de239200f0adf9af5c1b0162677e9"} Jul 02 14:17:11.283062 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.283028 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:17:11.411360 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.411326 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-bound-sa-token\") pod \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " Jul 02 14:17:11.411360 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.411358 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") pod \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " Jul 02 14:17:11.411591 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.411413 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-image-registry-private-configuration\") pod \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " Jul 02 14:17:11.411591 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.411434 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-installation-pull-secrets\") pod \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " Jul 02 14:17:11.411591 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.411484 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-trusted-ca\") pod \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " Jul 02 14:17:11.411703 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.411683 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np9zg\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-kube-api-access-np9zg\") pod \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " Jul 02 14:17:11.411760 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.411737 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-ca-trust-extracted\") pod \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " Jul 02 14:17:11.411814 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.411771 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-certificates\") pod \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\" (UID: \"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65\") " Jul 02 14:17:11.412167 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.412137 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:17:11.412401 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.412376 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:17:11.414251 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.414211 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:17:11.414563 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.414539 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:17:11.416644 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.416601 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:17:11.416802 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.416772 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:17:11.417076 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.417003 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-kube-api-access-np9zg" (OuterVolumeSpecName: "kube-api-access-np9zg") pod "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65"). InnerVolumeSpecName "kube-api-access-np9zg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:17:11.426840 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.426808 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" (UID: "8dc6dd9c-6110-413a-9ecf-4d414e7e5e65"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:17:11.516550 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.513131 2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-image-registry-private-configuration\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:11.516550 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.513178 2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-installation-pull-secrets\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:11.516550 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.513194 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-trusted-ca\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:11.516550 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.513209 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np9zg\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-kube-api-access-np9zg\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:11.516550 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.513223 2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-ca-trust-extracted\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:11.516550 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.513242 2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-certificates\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:11.516550 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.513256 2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-bound-sa-token\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:11.516550 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:11.513270 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65-registry-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:12.170333 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:12.170302 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65555b575-h4qjs" event={"ID":"8dc6dd9c-6110-413a-9ecf-4d414e7e5e65","Type":"ContainerDied","Data":"491a936feaac3b17dafc5d4e849ac19deffbcb6d2cd5acc0e14750ddd3c4a9cf"} Jul 02 14:17:12.170714 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:12.170358 2569 scope.go:117] "RemoveContainer" containerID="620d79c204b384144773f3907edcdd1b447de239200f0adf9af5c1b0162677e9" Jul 02 14:17:12.170714 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:12.170310 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65555b575-h4qjs" Jul 02 14:17:12.200821 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:12.200790 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65555b575-h4qjs"] Jul 02 14:17:12.208733 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:12.208702 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65555b575-h4qjs"] Jul 02 14:17:12.599950 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:12.599868 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" path="/var/lib/kubelet/pods/8dc6dd9c-6110-413a-9ecf-4d414e7e5e65/volumes" Jul 02 14:17:13.177657 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:13.177616 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerStarted","Data":"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea"} Jul 02 14:17:14.186635 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:14.186600 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerStarted","Data":"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e"} Jul 02 14:17:14.187018 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:14.186666 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerStarted","Data":"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0"} Jul 02 14:17:14.187018 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:14.186693 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerStarted","Data":"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88"} Jul 02 14:17:14.221483 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:14.221408 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.413815531 podStartE2EDuration="12.221385528s" totalImagesPullingTime="8.785236384s" totalInitContainerRuntime="22.333613ms" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:14.219163395 +0000 UTC m=+98.094246875" watchObservedRunningTime="2026-07-02 14:17:14.221385528 +0000 UTC m=+98.096468986" Jul 02 14:17:18.284843 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:18.284804 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:17:21.010550 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:21.010523 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rzbpn" Jul 02 14:17:22.231146 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.231113 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cd878c44c-928lt"] Jul 02 14:17:22.231547 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.231418 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" containerName="registry" Jul 02 14:17:22.231547 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.231430 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" containerName="registry" Jul 02 14:17:22.231547 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.231515 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dc6dd9c-6110-413a-9ecf-4d414e7e5e65" containerName="registry" Jul 02 14:17:22.296059 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.296017 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd878c44c-928lt"] Jul 02 14:17:22.296210 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.296139 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.308083 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.308062 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 14:17:22.405568 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.405536 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-config\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.405738 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.405576 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-oauth-config\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.405738 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.405616 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-serving-cert\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.405738 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.405636 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-service-ca\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.405738 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.405705 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-oauth-serving-cert\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.405873 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.405738 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d6xj\" (UniqueName: \"kubernetes.io/projected/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-kube-api-access-8d6xj\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.405873 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.405777 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-trusted-ca-bundle\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.507014 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.506929 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-oauth-serving-cert\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.507014 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.506983 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d6xj\" (UniqueName: \"kubernetes.io/projected/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-kube-api-access-8d6xj\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.507227 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.507035 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-trusted-ca-bundle\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.507227 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.507070 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-config\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.507227 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.507099 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-oauth-config\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.507227 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.507135 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-serving-cert\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.507227 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.507156 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-service-ca\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.507808 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.507780 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-oauth-serving-cert\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.507955 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.507927 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-service-ca\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.508120 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.507951 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-config\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.508187 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.507964 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-trusted-ca-bundle\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.509670 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.509641 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-oauth-config\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.509729 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.509692 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-serving-cert\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.525350 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.525323 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d6xj\" (UniqueName: \"kubernetes.io/projected/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-kube-api-access-8d6xj\") pod \"console-cd878c44c-928lt\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.605518 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.605494 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:22.773397 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:22.773320 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd878c44c-928lt"] Jul 02 14:17:22.776273 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:17:22.776245 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955f3c95_7dd2_4e33_8da2_2feb4b2ab2b1.slice/crio-08602efe474c8fd4a6ec50f5740e245cd1baae97552c37fa3938a8c7a833e608 WatchSource:0}: Error finding container 08602efe474c8fd4a6ec50f5740e245cd1baae97552c37fa3938a8c7a833e608: Status 404 returned error can't find the container with id 08602efe474c8fd4a6ec50f5740e245cd1baae97552c37fa3938a8c7a833e608 Jul 02 14:17:23.224264 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:23.224225 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd878c44c-928lt" event={"ID":"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1","Type":"ContainerStarted","Data":"dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e"} Jul 02 14:17:23.224431 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:23.224272 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd878c44c-928lt" event={"ID":"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1","Type":"ContainerStarted","Data":"08602efe474c8fd4a6ec50f5740e245cd1baae97552c37fa3938a8c7a833e608"} Jul 02 14:17:23.248745 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:23.248687 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cd878c44c-928lt" podStartSLOduration=1.248673225 podStartE2EDuration="1.248673225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:23.247161532 +0000 UTC m=+107.122244987" watchObservedRunningTime="2026-07-02 14:17:23.248673225 +0000 UTC m=+107.123756680" Jul 02 14:17:32.606262 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:32.606230 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:32.606712 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:32.606291 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:32.610900 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:32.610875 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:33.267351 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:33.267325 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:17:33.342001 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:33.341967 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695698fdf9-9jh5p"] Jul 02 14:17:58.363394 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.363320 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-695698fdf9-9jh5p" podUID="199d3194-744c-4470-b236-0afcbc4dfbee" containerName="console" containerID="cri-o://14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e" gracePeriod=15 Jul 02 14:17:58.624048 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.623986 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:17:58.719506 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.719471 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-serving-cert\") pod \"199d3194-744c-4470-b236-0afcbc4dfbee\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " Jul 02 14:17:58.719681 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.719553 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-oauth-config\") pod \"199d3194-744c-4470-b236-0afcbc4dfbee\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " Jul 02 14:17:58.719681 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.719592 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-service-ca\") pod \"199d3194-744c-4470-b236-0afcbc4dfbee\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " Jul 02 14:17:58.719681 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.719641 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-oauth-serving-cert\") pod \"199d3194-744c-4470-b236-0afcbc4dfbee\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " Jul 02 14:17:58.719681 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.719666 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcg6w\" (UniqueName: \"kubernetes.io/projected/199d3194-744c-4470-b236-0afcbc4dfbee-kube-api-access-jcg6w\") pod \"199d3194-744c-4470-b236-0afcbc4dfbee\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " Jul 02 14:17:58.719865 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.719686 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-console-config\") pod \"199d3194-744c-4470-b236-0afcbc4dfbee\" (UID: \"199d3194-744c-4470-b236-0afcbc4dfbee\") " Jul 02 14:17:58.720117 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.720069 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-service-ca" (OuterVolumeSpecName: "service-ca") pod "199d3194-744c-4470-b236-0afcbc4dfbee" (UID: "199d3194-744c-4470-b236-0afcbc4dfbee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:17:58.720315 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.720134 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-console-config" (OuterVolumeSpecName: "console-config") pod "199d3194-744c-4470-b236-0afcbc4dfbee" (UID: "199d3194-744c-4470-b236-0afcbc4dfbee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:17:58.720383 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.720336 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "199d3194-744c-4470-b236-0afcbc4dfbee" (UID: "199d3194-744c-4470-b236-0afcbc4dfbee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:17:58.721893 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.721864 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "199d3194-744c-4470-b236-0afcbc4dfbee" (UID: "199d3194-744c-4470-b236-0afcbc4dfbee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:17:58.721991 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.721922 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/199d3194-744c-4470-b236-0afcbc4dfbee-kube-api-access-jcg6w" (OuterVolumeSpecName: "kube-api-access-jcg6w") pod "199d3194-744c-4470-b236-0afcbc4dfbee" (UID: "199d3194-744c-4470-b236-0afcbc4dfbee"). InnerVolumeSpecName "kube-api-access-jcg6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:17:58.722033 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.722014 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "199d3194-744c-4470-b236-0afcbc4dfbee" (UID: "199d3194-744c-4470-b236-0afcbc4dfbee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:17:58.821247 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.821214 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-oauth-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:58.821247 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.821240 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-service-ca\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:58.821247 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.821250 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-oauth-serving-cert\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:58.821492 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.821259 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcg6w\" (UniqueName: \"kubernetes.io/projected/199d3194-744c-4470-b236-0afcbc4dfbee-kube-api-access-jcg6w\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:58.821492 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.821268 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/199d3194-744c-4470-b236-0afcbc4dfbee-console-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:58.821492 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:58.821276 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/199d3194-744c-4470-b236-0afcbc4dfbee-console-serving-cert\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:17:59.371189 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:59.371158 2569 generic.go:356] "Generic (PLEG): container finished" podID="199d3194-744c-4470-b236-0afcbc4dfbee" containerID="14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e" exitCode=0 Jul 02 14:17:59.371588 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:59.371204 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695698fdf9-9jh5p" Jul 02 14:17:59.371588 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:59.371216 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695698fdf9-9jh5p" event={"ID":"199d3194-744c-4470-b236-0afcbc4dfbee","Type":"ContainerDied","Data":"14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e"} Jul 02 14:17:59.371588 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:59.371258 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695698fdf9-9jh5p" event={"ID":"199d3194-744c-4470-b236-0afcbc4dfbee","Type":"ContainerDied","Data":"22479b32839446e4e4ae0eb149dc96dde5f3c18279822ff841d15ca28b783f64"} Jul 02 14:17:59.371588 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:59.371277 2569 scope.go:117] "RemoveContainer" containerID="14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e" Jul 02 14:17:59.380683 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:59.380663 2569 scope.go:117] "RemoveContainer" containerID="14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e" Jul 02 14:17:59.380958 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:17:59.380935 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e\": container with ID starting with 14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e not found: ID does not exist" containerID="14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e" Jul 02 14:17:59.381035 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:59.380966 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e"} err="failed to get container status \"14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e\": rpc error: code = NotFound desc = could not find container \"14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e\": container with ID starting with 14d1a0ba6d73f795f00a9b9a64f6aa8260c6dd72eadcc04e1a11ed6149353a4e not found: ID does not exist" Jul 02 14:17:59.392619 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:59.392595 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695698fdf9-9jh5p"] Jul 02 14:17:59.396576 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:17:59.396555 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-695698fdf9-9jh5p"] Jul 02 14:18:00.597723 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:00.597682 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="199d3194-744c-4470-b236-0afcbc4dfbee" path="/var/lib/kubelet/pods/199d3194-744c-4470-b236-0afcbc4dfbee/volumes" Jul 02 14:18:03.284604 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:03.284576 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:03.303620 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:03.303588 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:03.401949 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:03.401924 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:17.144727 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.144691 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 14:18:17.145205 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.145004 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="alertmanager" containerID="cri-o://a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f" gracePeriod=120 Jul 02 14:18:17.145205 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.145075 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy-metric" containerID="cri-o://40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef" gracePeriod=120 Jul 02 14:18:17.145205 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.145103 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy-web" containerID="cri-o://1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575" gracePeriod=120 Jul 02 14:18:17.145205 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.145131 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="prom-label-proxy" containerID="cri-o://f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b" gracePeriod=120 Jul 02 14:18:17.145205 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.145165 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy" containerID="cri-o://36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb" gracePeriod=120 Jul 02 14:18:17.145484 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.145102 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="config-reloader" containerID="cri-o://43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f" gracePeriod=120 Jul 02 14:18:17.440257 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.440174 2569 generic.go:356] "Generic (PLEG): container finished" podID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerID="f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b" exitCode=0 Jul 02 14:18:17.440257 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.440200 2569 generic.go:356] "Generic (PLEG): container finished" podID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerID="36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb" exitCode=0 Jul 02 14:18:17.440257 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.440206 2569 generic.go:356] "Generic (PLEG): container finished" podID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerID="43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f" exitCode=0 Jul 02 14:18:17.440257 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.440211 2569 generic.go:356] "Generic (PLEG): container finished" podID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerID="a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f" exitCode=0 Jul 02 14:18:17.440257 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.440241 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerDied","Data":"f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b"} Jul 02 14:18:17.440532 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.440302 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerDied","Data":"36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb"} Jul 02 14:18:17.440532 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.440329 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerDied","Data":"43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f"} Jul 02 14:18:17.440532 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:17.440358 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerDied","Data":"a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f"} Jul 02 14:18:18.395903 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.395875 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:18:18.449040 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.449004 2569 generic.go:356] "Generic (PLEG): container finished" podID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerID="40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef" exitCode=0 Jul 02 14:18:18.449040 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.449039 2569 generic.go:356] "Generic (PLEG): container finished" podID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerID="1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575" exitCode=0 Jul 02 14:18:18.449238 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.449099 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 14:18:18.449238 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.449092 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerDied","Data":"40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef"} Jul 02 14:18:18.449238 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.449226 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerDied","Data":"1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575"} Jul 02 14:18:18.449336 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.449275 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92c46c7b-c750-4501-a45f-b3e160dcffe5","Type":"ContainerDied","Data":"9f2d0fc712d34e40f7d1eaf0233c7cce4c029f936f0265161fb46dbec4819768"} Jul 02 14:18:18.449336 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.449298 2569 scope.go:117] "RemoveContainer" containerID="f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b" Jul 02 14:18:18.458015 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.457997 2569 scope.go:117] "RemoveContainer" containerID="40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef" Jul 02 14:18:18.465027 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.465007 2569 scope.go:117] "RemoveContainer" containerID="36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb" Jul 02 14:18:18.471658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.471640 2569 scope.go:117] "RemoveContainer" containerID="1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575" Jul 02 14:18:18.478044 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.478022 2569 scope.go:117] "RemoveContainer" containerID="43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f" Jul 02 14:18:18.485484 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.485467 2569 scope.go:117] "RemoveContainer" containerID="a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f" Jul 02 14:18:18.487519 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487501 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-main-tls\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.487605 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487528 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-volume\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.487605 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487552 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-cluster-tls-config\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.487693 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487607 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-metrics-client-ca\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.487693 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487652 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-trusted-ca-bundle\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.487693 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487687 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.487839 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487733 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-tls-assets\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.487839 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487771 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-main-db\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.487839 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487795 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-web-config\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.487839 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487818 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk79m\" (UniqueName: \"kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-kube-api-access-nk79m\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.488034 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487847 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.488034 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487885 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-out\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.488034 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487909 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-web\") pod \"92c46c7b-c750-4501-a45f-b3e160dcffe5\" (UID: \"92c46c7b-c750-4501-a45f-b3e160dcffe5\") " Jul 02 14:18:18.488034 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.487999 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:18.488221 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.488133 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:18.488221 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.488168 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:18:18.488587 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.488385 2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-metrics-client-ca\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.488587 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.488412 2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.488587 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.488458 2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-alertmanager-main-db\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.490924 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.490883 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-kube-api-access-nk79m" (OuterVolumeSpecName: "kube-api-access-nk79m") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "kube-api-access-nk79m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:18:18.491828 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.491798 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:18.492124 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.492086 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-out" (OuterVolumeSpecName: "config-out") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:18:18.492124 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.492089 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:18:18.492367 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.492340 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:18.492973 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.492930 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-volume" (OuterVolumeSpecName: "config-volume") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:18.492973 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.492945 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:18.492973 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.492958 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:18.494876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.494858 2569 scope.go:117] "RemoveContainer" containerID="dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d" Jul 02 14:18:18.495424 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.495404 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:18.502135 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.502110 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-web-config" (OuterVolumeSpecName: "web-config") pod "92c46c7b-c750-4501-a45f-b3e160dcffe5" (UID: "92c46c7b-c750-4501-a45f-b3e160dcffe5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:18.503668 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.503652 2569 scope.go:117] "RemoveContainer" containerID="f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b" Jul 02 14:18:18.503904 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:18.503886 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b\": container with ID starting with f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b not found: ID does not exist" containerID="f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b" Jul 02 14:18:18.503971 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.503917 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b"} err="failed to get container status \"f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b\": rpc error: code = NotFound desc = could not find container \"f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b\": container with ID starting with f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b not found: ID does not exist" Jul 02 14:18:18.503971 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.503944 2569 scope.go:117] "RemoveContainer" containerID="40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef" Jul 02 14:18:18.504214 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:18.504197 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef\": container with ID starting with 40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef not found: ID does not exist" containerID="40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef" Jul 02 14:18:18.504252 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.504219 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef"} err="failed to get container status \"40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef\": rpc error: code = NotFound desc = could not find container \"40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef\": container with ID starting with 40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef not found: ID does not exist" Jul 02 14:18:18.504252 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.504235 2569 scope.go:117] "RemoveContainer" containerID="36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb" Jul 02 14:18:18.504459 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:18.504431 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb\": container with ID starting with 36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb not found: ID does not exist" containerID="36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb" Jul 02 14:18:18.504549 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.504527 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb"} err="failed to get container status \"36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb\": rpc error: code = NotFound desc = could not find container \"36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb\": container with ID starting with 36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb not found: ID does not exist" Jul 02 14:18:18.504609 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.504553 2569 scope.go:117] "RemoveContainer" containerID="1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575" Jul 02 14:18:18.504818 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:18.504802 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575\": container with ID starting with 1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575 not found: ID does not exist" containerID="1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575" Jul 02 14:18:18.504856 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.504824 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575"} err="failed to get container status \"1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575\": rpc error: code = NotFound desc = could not find container \"1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575\": container with ID starting with 1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575 not found: ID does not exist" Jul 02 14:18:18.504856 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.504837 2569 scope.go:117] "RemoveContainer" containerID="43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f" Jul 02 14:18:18.505050 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:18.505034 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f\": container with ID starting with 43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f not found: ID does not exist" containerID="43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f" Jul 02 14:18:18.505091 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.505055 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f"} err="failed to get container status \"43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f\": rpc error: code = NotFound desc = could not find container \"43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f\": container with ID starting with 43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f not found: ID does not exist" Jul 02 14:18:18.505091 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.505070 2569 scope.go:117] "RemoveContainer" containerID="a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f" Jul 02 14:18:18.505277 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:18.505261 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f\": container with ID starting with a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f not found: ID does not exist" containerID="a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f" Jul 02 14:18:18.505317 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.505281 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f"} err="failed to get container status \"a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f\": rpc error: code = NotFound desc = could not find container \"a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f\": container with ID starting with a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f not found: ID does not exist" Jul 02 14:18:18.505317 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.505294 2569 scope.go:117] "RemoveContainer" containerID="dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d" Jul 02 14:18:18.505510 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:18.505495 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d\": container with ID starting with dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d not found: ID does not exist" containerID="dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d" Jul 02 14:18:18.505567 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.505512 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d"} err="failed to get container status \"dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d\": rpc error: code = NotFound desc = could not find container \"dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d\": container with ID starting with dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d not found: ID does not exist" Jul 02 14:18:18.505567 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.505523 2569 scope.go:117] "RemoveContainer" containerID="f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b" Jul 02 14:18:18.505719 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.505698 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b"} err="failed to get container status \"f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b\": rpc error: code = NotFound desc = could not find container \"f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b\": container with ID starting with f2913413dfecd30abc6953becfbdd4abc93669c66794d0226f155b6f3b2c469b not found: ID does not exist" Jul 02 14:18:18.505763 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.505720 2569 scope.go:117] "RemoveContainer" containerID="40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef" Jul 02 14:18:18.505905 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.505891 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef"} err="failed to get container status \"40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef\": rpc error: code = NotFound desc = could not find container \"40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef\": container with ID starting with 40eb52c5c01c7c938d5ba146f8fbfa0e22ee5f31cc25b3c5b05524074fa12eef not found: ID does not exist" Jul 02 14:18:18.505947 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.505905 2569 scope.go:117] "RemoveContainer" containerID="36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb" Jul 02 14:18:18.506108 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.506092 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb"} err="failed to get container status \"36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb\": rpc error: code = NotFound desc = could not find container \"36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb\": container with ID starting with 36bf37bf721afd67768ce4af956256d0c9ea09c64a97448d5af5c1e9fbd3efcb not found: ID does not exist" Jul 02 14:18:18.506159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.506108 2569 scope.go:117] "RemoveContainer" containerID="1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575" Jul 02 14:18:18.506290 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.506272 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575"} err="failed to get container status \"1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575\": rpc error: code = NotFound desc = could not find container \"1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575\": container with ID starting with 1fcb3b0452a7de2d878f432ce5b8f96b9df41f03b0623c49a47c66fb5b061575 not found: ID does not exist" Jul 02 14:18:18.506369 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.506290 2569 scope.go:117] "RemoveContainer" containerID="43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f" Jul 02 14:18:18.506487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.506467 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f"} err="failed to get container status \"43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f\": rpc error: code = NotFound desc = could not find container \"43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f\": container with ID starting with 43e208d62b41f55b74ac04298f02bf1946b87495b39d2490ae528d30ed51e10f not found: ID does not exist" Jul 02 14:18:18.506544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.506488 2569 scope.go:117] "RemoveContainer" containerID="a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f" Jul 02 14:18:18.506694 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.506677 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f"} err="failed to get container status \"a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f\": rpc error: code = NotFound desc = could not find container \"a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f\": container with ID starting with a3fa6c3e6997f1de98487c13e95511a0a9bc98e38f6d100e881456213f9f0f7f not found: ID does not exist" Jul 02 14:18:18.506740 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.506694 2569 scope.go:117] "RemoveContainer" containerID="dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d" Jul 02 14:18:18.506871 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.506858 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d"} err="failed to get container status \"dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d\": rpc error: code = NotFound desc = could not find container \"dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d\": container with ID starting with dac75f49ae1c8cfcb8b2365075a669bcc72e0be0913b2f4e02c54f17aaa6ab1d not found: ID does not exist" Jul 02 14:18:18.589625 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.589592 2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-web-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.589625 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.589626 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk79m\" (UniqueName: \"kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-kube-api-access-nk79m\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.589817 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.589643 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.589817 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.589658 2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-out\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.589817 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.589674 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.589817 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.589688 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-main-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.589817 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.589703 2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-config-volume\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.589817 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.589724 2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-cluster-tls-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.589817 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.589736 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/92c46c7b-c750-4501-a45f-b3e160dcffe5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.589817 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.589748 2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92c46c7b-c750-4501-a45f-b3e160dcffe5-tls-assets\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:18.768005 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.767927 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 14:18:18.772548 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:18.772518 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 14:18:20.597259 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:20.597224 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" path="/var/lib/kubelet/pods/92c46c7b-c750-4501-a45f-b3e160dcffe5/volumes" Jul 02 14:18:21.375854 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.375809 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 14:18:21.376285 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.376121 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="prometheus" containerID="cri-o://ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe" gracePeriod=600 Jul 02 14:18:21.376285 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.376160 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy" containerID="cri-o://2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0" gracePeriod=600 Jul 02 14:18:21.376285 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.376160 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="thanos-sidecar" containerID="cri-o://49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea" gracePeriod=600 Jul 02 14:18:21.376285 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.376191 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="config-reloader" containerID="cri-o://e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63" gracePeriod=600 Jul 02 14:18:21.376285 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.376191 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e" gracePeriod=600 Jul 02 14:18:21.376285 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.376174 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy-web" containerID="cri-o://c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88" gracePeriod=600 Jul 02 14:18:21.642685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.642659 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:21.716659 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716623 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-serving-certs-ca-bundle\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.716803 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716672 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-db\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.716803 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716709 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-kube-rbac-proxy\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.716803 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716730 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config-out\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.716803 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716752 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-2\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.716803 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716771 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-tls-assets\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.716803 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716793 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716830 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-web-config\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716854 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-grpc-tls\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716879 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-tls\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716905 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-1\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716927 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-trusted-ca-bundle\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716945 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-thanos-prometheus-http-client-file\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716968 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shhx7\" (UniqueName: \"kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-kube-api-access-shhx7\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.716996 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-metrics-client-ca\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.717033 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-kubelet-serving-ca-bundle\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717553 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.717135 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:21.717553 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.717170 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717553 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.717221 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-0\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717553 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.717259 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-metrics-client-certs\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.717799 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.717774 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:21.718099 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.718067 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:21.718194 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.718120 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:21.718342 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.718317 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:18:21.718406 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.718394 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config\") pod \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\" (UID: \"c17167f2-8b83-4a1f-b29f-b457cf498d0c\") " Jul 02 14:18:21.718700 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.718680 2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.718792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.718707 2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.718792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.718701 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:21.718792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.718723 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-db\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.718792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.718739 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.718792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.718755 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.719650 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.719615 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:21.719784 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.719763 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:21.720066 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.720042 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:21.720934 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.720899 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:21.721158 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.721116 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:21.721871 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.721843 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-kube-api-access-shhx7" (OuterVolumeSpecName: "kube-api-access-shhx7") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "kube-api-access-shhx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:18:21.721962 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.721907 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config" (OuterVolumeSpecName: "config") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:21.722177 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.722138 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config-out" (OuterVolumeSpecName: "config-out") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:18:21.722267 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.722202 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:21.722487 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.722462 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:21.722550 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.722496 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:21.723112 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.723091 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:21.724042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.724020 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:18:21.729874 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.729851 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-web-config" (OuterVolumeSpecName: "web-config") pod "c17167f2-8b83-4a1f-b29f-b457cf498d0c" (UID: "c17167f2-8b83-4a1f-b29f-b457cf498d0c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:21.819375 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819343 2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-kube-rbac-proxy\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819375 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819371 2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config-out\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819375 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819381 2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-tls-assets\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819391 2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819401 2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-web-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819408 2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-grpc-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819416 2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819425 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819434 2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819442 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shhx7\" (UniqueName: \"kubernetes.io/projected/c17167f2-8b83-4a1f-b29f-b457cf498d0c-kube-api-access-shhx7\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819467 2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-configmap-metrics-client-ca\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819476 2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819484 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c17167f2-8b83-4a1f-b29f-b457cf498d0c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819492 2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-secret-metrics-client-certs\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:21.819607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:21.819500 2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c17167f2-8b83-4a1f-b29f-b457cf498d0c-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:18:22.469792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469761 2569 generic.go:356] "Generic (PLEG): container finished" podID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerID="c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e" exitCode=0 Jul 02 14:18:22.469792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469788 2569 generic.go:356] "Generic (PLEG): container finished" podID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerID="2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0" exitCode=0 Jul 02 14:18:22.469792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469797 2569 generic.go:356] "Generic (PLEG): container finished" podID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerID="c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88" exitCode=0 Jul 02 14:18:22.469792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469803 2569 generic.go:356] "Generic (PLEG): container finished" podID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerID="49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea" exitCode=0 Jul 02 14:18:22.470042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469808 2569 generic.go:356] "Generic (PLEG): container finished" podID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerID="e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63" exitCode=0 Jul 02 14:18:22.470042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469815 2569 generic.go:356] "Generic (PLEG): container finished" podID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerID="ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe" exitCode=0 Jul 02 14:18:22.470042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469846 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.470042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469848 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerDied","Data":"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e"} Jul 02 14:18:22.470042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469939 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerDied","Data":"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0"} Jul 02 14:18:22.470042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469966 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerDied","Data":"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88"} Jul 02 14:18:22.470042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.469993 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerDied","Data":"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea"} Jul 02 14:18:22.470042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.470011 2569 scope.go:117] "RemoveContainer" containerID="c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e" Jul 02 14:18:22.470042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.470022 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerDied","Data":"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63"} Jul 02 14:18:22.470042 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.470047 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerDied","Data":"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe"} Jul 02 14:18:22.470381 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.470076 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c17167f2-8b83-4a1f-b29f-b457cf498d0c","Type":"ContainerDied","Data":"f490da13a56122b0e904eed84ea09424cf20619a90994a4a67001e1dce8d4a23"} Jul 02 14:18:22.477922 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.477853 2569 scope.go:117] "RemoveContainer" containerID="2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0" Jul 02 14:18:22.485658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.485642 2569 scope.go:117] "RemoveContainer" containerID="c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88" Jul 02 14:18:22.493582 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.493321 2569 scope.go:117] "RemoveContainer" containerID="49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea" Jul 02 14:18:22.497904 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.497870 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 14:18:22.502593 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.502260 2569 scope.go:117] "RemoveContainer" containerID="e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63" Jul 02 14:18:22.504950 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.504930 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 14:18:22.510154 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.510139 2569 scope.go:117] "RemoveContainer" containerID="ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe" Jul 02 14:18:22.516949 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.516932 2569 scope.go:117] "RemoveContainer" containerID="7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d" Jul 02 14:18:22.523545 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.523527 2569 scope.go:117] "RemoveContainer" containerID="c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e" Jul 02 14:18:22.523816 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:22.523797 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": container with ID starting with c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e not found: ID does not exist" containerID="c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e" Jul 02 14:18:22.523867 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.523823 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e"} err="failed to get container status \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": rpc error: code = NotFound desc = could not find container \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": container with ID starting with c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e not found: ID does not exist" Jul 02 14:18:22.523867 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.523842 2569 scope.go:117] "RemoveContainer" containerID="2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0" Jul 02 14:18:22.524047 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:22.524031 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": container with ID starting with 2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0 not found: ID does not exist" containerID="2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0" Jul 02 14:18:22.524082 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.524052 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0"} err="failed to get container status \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": rpc error: code = NotFound desc = could not find container \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": container with ID starting with 2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0 not found: ID does not exist" Jul 02 14:18:22.524082 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.524067 2569 scope.go:117] "RemoveContainer" containerID="c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88" Jul 02 14:18:22.524280 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:22.524267 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": container with ID starting with c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88 not found: ID does not exist" containerID="c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88" Jul 02 14:18:22.524317 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.524282 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88"} err="failed to get container status \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": rpc error: code = NotFound desc = could not find container \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": container with ID starting with c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88 not found: ID does not exist" Jul 02 14:18:22.524317 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.524293 2569 scope.go:117] "RemoveContainer" containerID="49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea" Jul 02 14:18:22.524524 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:22.524502 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": container with ID starting with 49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea not found: ID does not exist" containerID="49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea" Jul 02 14:18:22.524616 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.524530 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea"} err="failed to get container status \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": rpc error: code = NotFound desc = could not find container \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": container with ID starting with 49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea not found: ID does not exist" Jul 02 14:18:22.524616 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.524548 2569 scope.go:117] "RemoveContainer" containerID="e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63" Jul 02 14:18:22.524816 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:22.524788 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": container with ID starting with e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63 not found: ID does not exist" containerID="e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63" Jul 02 14:18:22.524888 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.524827 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63"} err="failed to get container status \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": rpc error: code = NotFound desc = could not find container \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": container with ID starting with e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63 not found: ID does not exist" Jul 02 14:18:22.524888 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.524847 2569 scope.go:117] "RemoveContainer" containerID="ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe" Jul 02 14:18:22.525220 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:22.525199 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": container with ID starting with ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe not found: ID does not exist" containerID="ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe" Jul 02 14:18:22.525267 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.525226 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe"} err="failed to get container status \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": rpc error: code = NotFound desc = could not find container \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": container with ID starting with ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe not found: ID does not exist" Jul 02 14:18:22.525267 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.525239 2569 scope.go:117] "RemoveContainer" containerID="7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d" Jul 02 14:18:22.525493 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:18:22.525472 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": container with ID starting with 7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d not found: ID does not exist" containerID="7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d" Jul 02 14:18:22.525551 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.525498 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d"} err="failed to get container status \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": rpc error: code = NotFound desc = could not find container \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": container with ID starting with 7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d not found: ID does not exist" Jul 02 14:18:22.525551 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.525515 2569 scope.go:117] "RemoveContainer" containerID="c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e" Jul 02 14:18:22.525790 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.525771 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e"} err="failed to get container status \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": rpc error: code = NotFound desc = could not find container \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": container with ID starting with c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e not found: ID does not exist" Jul 02 14:18:22.525833 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.525790 2569 scope.go:117] "RemoveContainer" containerID="2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0" Jul 02 14:18:22.526018 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.526000 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0"} err="failed to get container status \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": rpc error: code = NotFound desc = could not find container \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": container with ID starting with 2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0 not found: ID does not exist" Jul 02 14:18:22.526065 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.526019 2569 scope.go:117] "RemoveContainer" containerID="c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88" Jul 02 14:18:22.526247 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.526232 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88"} err="failed to get container status \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": rpc error: code = NotFound desc = could not find container \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": container with ID starting with c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88 not found: ID does not exist" Jul 02 14:18:22.526286 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.526254 2569 scope.go:117] "RemoveContainer" containerID="49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea" Jul 02 14:18:22.526500 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.526481 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea"} err="failed to get container status \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": rpc error: code = NotFound desc = could not find container \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": container with ID starting with 49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea not found: ID does not exist" Jul 02 14:18:22.526552 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.526501 2569 scope.go:117] "RemoveContainer" containerID="e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63" Jul 02 14:18:22.526715 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.526698 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63"} err="failed to get container status \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": rpc error: code = NotFound desc = could not find container \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": container with ID starting with e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63 not found: ID does not exist" Jul 02 14:18:22.526761 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.526716 2569 scope.go:117] "RemoveContainer" containerID="ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe" Jul 02 14:18:22.526947 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.526930 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe"} err="failed to get container status \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": rpc error: code = NotFound desc = could not find container \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": container with ID starting with ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe not found: ID does not exist" Jul 02 14:18:22.526991 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.526948 2569 scope.go:117] "RemoveContainer" containerID="7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d" Jul 02 14:18:22.527159 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.527129 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d"} err="failed to get container status \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": rpc error: code = NotFound desc = could not find container \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": container with ID starting with 7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d not found: ID does not exist" Jul 02 14:18:22.527228 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.527162 2569 scope.go:117] "RemoveContainer" containerID="c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e" Jul 02 14:18:22.527386 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.527369 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e"} err="failed to get container status \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": rpc error: code = NotFound desc = could not find container \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": container with ID starting with c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e not found: ID does not exist" Jul 02 14:18:22.527429 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.527387 2569 scope.go:117] "RemoveContainer" containerID="2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0" Jul 02 14:18:22.527598 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.527578 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0"} err="failed to get container status \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": rpc error: code = NotFound desc = could not find container \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": container with ID starting with 2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0 not found: ID does not exist" Jul 02 14:18:22.527598 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.527599 2569 scope.go:117] "RemoveContainer" containerID="c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88" Jul 02 14:18:22.527785 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.527758 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88"} err="failed to get container status \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": rpc error: code = NotFound desc = could not find container \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": container with ID starting with c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88 not found: ID does not exist" Jul 02 14:18:22.527853 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.527786 2569 scope.go:117] "RemoveContainer" containerID="49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea" Jul 02 14:18:22.528018 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.527998 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea"} err="failed to get container status \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": rpc error: code = NotFound desc = could not find container \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": container with ID starting with 49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea not found: ID does not exist" Jul 02 14:18:22.528068 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.528018 2569 scope.go:117] "RemoveContainer" containerID="e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63" Jul 02 14:18:22.528228 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.528211 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63"} err="failed to get container status \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": rpc error: code = NotFound desc = could not find container \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": container with ID starting with e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63 not found: ID does not exist" Jul 02 14:18:22.528272 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.528228 2569 scope.go:117] "RemoveContainer" containerID="ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe" Jul 02 14:18:22.528471 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.528424 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe"} err="failed to get container status \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": rpc error: code = NotFound desc = could not find container \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": container with ID starting with ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe not found: ID does not exist" Jul 02 14:18:22.528566 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.528516 2569 scope.go:117] "RemoveContainer" containerID="7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d" Jul 02 14:18:22.528735 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.528716 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d"} err="failed to get container status \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": rpc error: code = NotFound desc = could not find container \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": container with ID starting with 7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d not found: ID does not exist" Jul 02 14:18:22.528809 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.528736 2569 scope.go:117] "RemoveContainer" containerID="c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e" Jul 02 14:18:22.528943 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.528925 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e"} err="failed to get container status \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": rpc error: code = NotFound desc = could not find container \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": container with ID starting with c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e not found: ID does not exist" Jul 02 14:18:22.528995 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.528944 2569 scope.go:117] "RemoveContainer" containerID="2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0" Jul 02 14:18:22.529162 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.529146 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0"} err="failed to get container status \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": rpc error: code = NotFound desc = could not find container \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": container with ID starting with 2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0 not found: ID does not exist" Jul 02 14:18:22.529207 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.529162 2569 scope.go:117] "RemoveContainer" containerID="c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88" Jul 02 14:18:22.529357 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.529335 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88"} err="failed to get container status \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": rpc error: code = NotFound desc = could not find container \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": container with ID starting with c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88 not found: ID does not exist" Jul 02 14:18:22.529425 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.529359 2569 scope.go:117] "RemoveContainer" containerID="49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea" Jul 02 14:18:22.529618 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.529602 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea"} err="failed to get container status \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": rpc error: code = NotFound desc = could not find container \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": container with ID starting with 49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea not found: ID does not exist" Jul 02 14:18:22.529669 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.529618 2569 scope.go:117] "RemoveContainer" containerID="e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63" Jul 02 14:18:22.529830 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.529813 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63"} err="failed to get container status \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": rpc error: code = NotFound desc = could not find container \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": container with ID starting with e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63 not found: ID does not exist" Jul 02 14:18:22.529874 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.529831 2569 scope.go:117] "RemoveContainer" containerID="ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe" Jul 02 14:18:22.530072 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.530054 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe"} err="failed to get container status \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": rpc error: code = NotFound desc = could not find container \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": container with ID starting with ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe not found: ID does not exist" Jul 02 14:18:22.530136 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.530073 2569 scope.go:117] "RemoveContainer" containerID="7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d" Jul 02 14:18:22.530286 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.530268 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d"} err="failed to get container status \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": rpc error: code = NotFound desc = could not find container \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": container with ID starting with 7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d not found: ID does not exist" Jul 02 14:18:22.530335 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.530288 2569 scope.go:117] "RemoveContainer" containerID="c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e" Jul 02 14:18:22.530523 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.530506 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e"} err="failed to get container status \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": rpc error: code = NotFound desc = could not find container \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": container with ID starting with c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e not found: ID does not exist" Jul 02 14:18:22.530584 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.530524 2569 scope.go:117] "RemoveContainer" containerID="2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0" Jul 02 14:18:22.530761 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.530745 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0"} err="failed to get container status \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": rpc error: code = NotFound desc = could not find container \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": container with ID starting with 2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0 not found: ID does not exist" Jul 02 14:18:22.530809 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.530762 2569 scope.go:117] "RemoveContainer" containerID="c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88" Jul 02 14:18:22.530983 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.530966 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88"} err="failed to get container status \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": rpc error: code = NotFound desc = could not find container \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": container with ID starting with c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88 not found: ID does not exist" Jul 02 14:18:22.531035 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.530983 2569 scope.go:117] "RemoveContainer" containerID="49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea" Jul 02 14:18:22.531195 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.531178 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea"} err="failed to get container status \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": rpc error: code = NotFound desc = could not find container \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": container with ID starting with 49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea not found: ID does not exist" Jul 02 14:18:22.531253 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.531196 2569 scope.go:117] "RemoveContainer" containerID="e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63" Jul 02 14:18:22.531437 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.531419 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63"} err="failed to get container status \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": rpc error: code = NotFound desc = could not find container \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": container with ID starting with e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63 not found: ID does not exist" Jul 02 14:18:22.531507 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.531438 2569 scope.go:117] "RemoveContainer" containerID="ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe" Jul 02 14:18:22.531663 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.531646 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe"} err="failed to get container status \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": rpc error: code = NotFound desc = could not find container \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": container with ID starting with ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe not found: ID does not exist" Jul 02 14:18:22.531715 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.531664 2569 scope.go:117] "RemoveContainer" containerID="7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d" Jul 02 14:18:22.531846 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.531826 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d"} err="failed to get container status \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": rpc error: code = NotFound desc = could not find container \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": container with ID starting with 7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d not found: ID does not exist" Jul 02 14:18:22.531889 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.531849 2569 scope.go:117] "RemoveContainer" containerID="c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e" Jul 02 14:18:22.532066 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.532038 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e"} err="failed to get container status \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": rpc error: code = NotFound desc = could not find container \"c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e\": container with ID starting with c25cf1ba094413aeab602facd1d5c890915cb1c59f8f3bf80173122ef4b0291e not found: ID does not exist" Jul 02 14:18:22.532066 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.532064 2569 scope.go:117] "RemoveContainer" containerID="2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0" Jul 02 14:18:22.532324 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.532292 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0"} err="failed to get container status \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": rpc error: code = NotFound desc = could not find container \"2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0\": container with ID starting with 2d86379b8d80799a8cf10499b280649735412dac1a1167368d1d39be102114a0 not found: ID does not exist" Jul 02 14:18:22.532324 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.532321 2569 scope.go:117] "RemoveContainer" containerID="c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88" Jul 02 14:18:22.533211 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.533190 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88"} err="failed to get container status \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": rpc error: code = NotFound desc = could not find container \"c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88\": container with ID starting with c7b4e4804511c4f08d9f083db83e7ad07b1b213eda8bb1e7bea56ea4edeeff88 not found: ID does not exist" Jul 02 14:18:22.533303 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.533211 2569 scope.go:117] "RemoveContainer" containerID="49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea" Jul 02 14:18:22.533507 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.533490 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea"} err="failed to get container status \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": rpc error: code = NotFound desc = could not find container \"49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea\": container with ID starting with 49f6624e7df1839607a5dbd09ea1ba375c2a734db1f838169a741e00a0477fea not found: ID does not exist" Jul 02 14:18:22.533507 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.533508 2569 scope.go:117] "RemoveContainer" containerID="e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63" Jul 02 14:18:22.533765 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.533744 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63"} err="failed to get container status \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": rpc error: code = NotFound desc = could not find container \"e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63\": container with ID starting with e11ff932a1f292b4e80c297448de1b763ef441c6b531c22a3867233c393c0d63 not found: ID does not exist" Jul 02 14:18:22.533842 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.533766 2569 scope.go:117] "RemoveContainer" containerID="ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe" Jul 02 14:18:22.534043 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534026 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe"} err="failed to get container status \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": rpc error: code = NotFound desc = could not find container \"ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe\": container with ID starting with ab5e9e6159020df2998885fe23295ddbe2c3b2f242a8555d18fe272270a963fe not found: ID does not exist" Jul 02 14:18:22.534095 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534045 2569 scope.go:117] "RemoveContainer" containerID="7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d" Jul 02 14:18:22.534258 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534240 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d"} err="failed to get container status \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": rpc error: code = NotFound desc = could not find container \"7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d\": container with ID starting with 7e0a5468377b3b644e6ba12ebcf32a09df4c53b706225639b0b27082f5782d0d not found: ID does not exist" Jul 02 14:18:22.534700 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534682 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 14:18:22.534913 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534904 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy-metric" Jul 02 14:18:22.534949 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534915 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy-metric" Jul 02 14:18:22.534949 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534924 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="thanos-sidecar" Jul 02 14:18:22.534949 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534929 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="thanos-sidecar" Jul 02 14:18:22.534949 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534938 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy" Jul 02 14:18:22.534949 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534944 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy" Jul 02 14:18:22.534949 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534951 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="init-config-reloader" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534957 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="init-config-reloader" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534962 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="config-reloader" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534967 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="config-reloader" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534975 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy-thanos" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534980 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy-thanos" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534986 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="init-config-reloader" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534991 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="init-config-reloader" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.534997 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy-web" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535002 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy-web" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535009 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="config-reloader" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535014 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="config-reloader" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535021 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="alertmanager" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535026 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="alertmanager" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535034 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="prom-label-proxy" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535039 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="prom-label-proxy" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535046 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="prometheus" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535051 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="prometheus" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535058 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy-web" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535077 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy-web" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535085 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535090 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535096 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="199d3194-744c-4470-b236-0afcbc4dfbee" containerName="console" Jul 02 14:18:22.535121 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535102 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="199d3194-744c-4470-b236-0afcbc4dfbee" containerName="console" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535141 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535148 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="config-reloader" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535154 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy-thanos" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535160 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy-web" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535165 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="prom-label-proxy" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535170 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="prometheus" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535175 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy-web" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535181 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="thanos-sidecar" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535186 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="kube-rbac-proxy-metric" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535190 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="config-reloader" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535195 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" containerName="kube-rbac-proxy" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535200 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="199d3194-744c-4470-b236-0afcbc4dfbee" containerName="console" Jul 02 14:18:22.535771 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.535205 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c46c7b-c750-4501-a45f-b3e160dcffe5" containerName="alertmanager" Jul 02 14:18:22.540767 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.540752 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.546004 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.545987 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 14:18:22.547941 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.547920 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 14:18:22.548023 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.547965 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 14:18:22.548259 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.548245 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2fdt34uv0jdvu\"" Jul 02 14:18:22.549275 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.549258 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 14:18:22.549907 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.549888 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 14:18:22.552177 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.552156 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 14:18:22.552177 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.552158 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 14:18:22.552401 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.552201 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 14:18:22.552473 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.552409 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 14:18:22.554039 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.554025 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-d7xbv\"" Jul 02 14:18:22.554527 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.554514 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 14:18:22.554926 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.554915 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 14:18:22.557526 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.557513 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 14:18:22.564018 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.563997 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 14:18:22.568548 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.568533 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 14:18:22.571086 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.571071 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 14:18:22.588277 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.588250 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 14:18:22.597633 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.597610 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c17167f2-8b83-4a1f-b29f-b457cf498d0c" path="/var/lib/kubelet/pods/c17167f2-8b83-4a1f-b29f-b457cf498d0c/volumes" Jul 02 14:18:22.625960 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.625937 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626071 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.625966 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626071 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.625990 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626071 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626007 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626192 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626084 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-config\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626192 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626112 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626192 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626138 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626192 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626155 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626333 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626204 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626333 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626229 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626333 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626251 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626333 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626276 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626333 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626306 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626333 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626324 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626565 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626341 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmf9c\" (UniqueName: \"kubernetes.io/projected/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-kube-api-access-cmf9c\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626565 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626363 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626565 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626395 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626565 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626415 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626565 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626467 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.626565 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.626489 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.727405 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727316 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.727405 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727359 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.727405 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727378 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.727405 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727398 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727414 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmf9c\" (UniqueName: \"kubernetes.io/projected/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-kube-api-access-cmf9c\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727567 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727620 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727656 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727687 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727708 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727741 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727767 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727794 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727843 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727888 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-config\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727912 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727941 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727964 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.727967 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.728006 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.728069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.728035 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.729532 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.728770 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.729532 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.728805 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.729532 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.729196 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.729532 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.729325 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.730073 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.729787 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.731009 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.730786 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.731009 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.730973 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.731009 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.731002 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.731209 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.731048 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.731419 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.731396 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.731909 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.731884 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.732687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.732620 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.732687 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.732638 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.732846 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.732729 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.732846 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.732818 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-config\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.733384 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.733368 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.733757 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.733742 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.734149 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.734127 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.736497 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.736478 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmf9c\" (UniqueName: \"kubernetes.io/projected/7b8d0e20-0504-4ccd-ae6b-8801d93ea920-kube-api-access-cmf9c\") pod \"prometheus-k8s-0\" (UID: \"7b8d0e20-0504-4ccd-ae6b-8801d93ea920\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.849391 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.849353 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:22.989410 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:22.989329 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 14:18:22.992469 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:18:22.992423 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b8d0e20_0504_4ccd_ae6b_8801d93ea920.slice/crio-e3c67dd9f5c648c5ddf493b581ca7edd3411cecab74aaa57efabe6ab6b1a372b WatchSource:0}: Error finding container e3c67dd9f5c648c5ddf493b581ca7edd3411cecab74aaa57efabe6ab6b1a372b: Status 404 returned error can't find the container with id e3c67dd9f5c648c5ddf493b581ca7edd3411cecab74aaa57efabe6ab6b1a372b Jul 02 14:18:23.475807 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:23.475774 2569 generic.go:356] "Generic (PLEG): container finished" podID="7b8d0e20-0504-4ccd-ae6b-8801d93ea920" containerID="ae44b2a35095e8ce3d155950caef4354f07187d1d99a48b147920aa447b16a42" exitCode=0 Jul 02 14:18:23.475973 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:23.475828 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b8d0e20-0504-4ccd-ae6b-8801d93ea920","Type":"ContainerDied","Data":"ae44b2a35095e8ce3d155950caef4354f07187d1d99a48b147920aa447b16a42"} Jul 02 14:18:23.475973 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:23.475876 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b8d0e20-0504-4ccd-ae6b-8801d93ea920","Type":"ContainerStarted","Data":"e3c67dd9f5c648c5ddf493b581ca7edd3411cecab74aaa57efabe6ab6b1a372b"} Jul 02 14:18:24.483357 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:24.483322 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b8d0e20-0504-4ccd-ae6b-8801d93ea920","Type":"ContainerStarted","Data":"8fe4dd4462793bf6b6e72db59048d4225d68748786217f60f040f57b80b33654"} Jul 02 14:18:24.483745 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:24.483391 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b8d0e20-0504-4ccd-ae6b-8801d93ea920","Type":"ContainerStarted","Data":"4bc238cecc6586a84b853f06a3f2d58a224622c7b58e60ef363b48f2a2de5977"} Jul 02 14:18:24.483745 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:24.483422 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b8d0e20-0504-4ccd-ae6b-8801d93ea920","Type":"ContainerStarted","Data":"23b7ce02e6e5847bbd27969570e6312849a69755a330c2afbe182ab6ab47ee8e"} Jul 02 14:18:24.483745 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:24.483462 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b8d0e20-0504-4ccd-ae6b-8801d93ea920","Type":"ContainerStarted","Data":"2178b49d0813b403a9a0fb50f3f801bd495acb92ad3d249fc07868a5f0783c8b"} Jul 02 14:18:24.483745 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:24.483502 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b8d0e20-0504-4ccd-ae6b-8801d93ea920","Type":"ContainerStarted","Data":"81b7fbdbe81aea98fe5facdd86bdde78a54fa19cdee855ec1a220962a1a33db5"} Jul 02 14:18:24.483745 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:24.483529 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b8d0e20-0504-4ccd-ae6b-8801d93ea920","Type":"ContainerStarted","Data":"2345d8f4a14fe91462b3ecbdc3788331a3f8f1b3dc10fbf5b9aafe5c93dff26e"} Jul 02 14:18:24.518651 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:24.518597 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.500147148 podStartE2EDuration="2.518580511s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.433363ms" isStatefulPod=false podCreationTimestamp="2026-07-02 14:18:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:24.516092787 +0000 UTC m=+168.391176267" watchObservedRunningTime="2026-07-02 14:18:24.518580511 +0000 UTC m=+168.393663965" Jul 02 14:18:27.849976 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:27.849926 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:18:37.574115 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:18:37.574081 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd878c44c-928lt"] Jul 02 14:19:02.593887 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.593838 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-cd878c44c-928lt" podUID="955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" containerName="console" containerID="cri-o://dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e" gracePeriod=15 Jul 02 14:19:02.833379 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.833356 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:19:02.847785 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.847744 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-service-ca\") pod \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " Jul 02 14:19:02.847862 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.847816 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-trusted-ca-bundle\") pod \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " Jul 02 14:19:02.847862 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.847844 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-oauth-config\") pod \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " Jul 02 14:19:02.847935 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.847874 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d6xj\" (UniqueName: \"kubernetes.io/projected/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-kube-api-access-8d6xj\") pod \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " Jul 02 14:19:02.848040 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.848021 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-oauth-serving-cert\") pod \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " Jul 02 14:19:02.848113 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.848065 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-config\") pod \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " Jul 02 14:19:02.848170 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.848117 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-serving-cert\") pod \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\" (UID: \"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1\") " Jul 02 14:19:02.848170 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.848143 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-service-ca" (OuterVolumeSpecName: "service-ca") pod "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" (UID: "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:19:02.848358 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.848256 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" (UID: "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:19:02.848408 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.848355 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-service-ca\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:19:02.852719 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.848795 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" (UID: "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:19:02.852719 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.848941 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-config" (OuterVolumeSpecName: "console-config") pod "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" (UID: "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:19:02.855663 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.855637 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-kube-api-access-8d6xj" (OuterVolumeSpecName: "kube-api-access-8d6xj") pod "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" (UID: "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1"). InnerVolumeSpecName "kube-api-access-8d6xj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:19:02.855933 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.855900 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" (UID: "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:19:02.856017 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.855956 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" (UID: "955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:19:02.949031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.948996 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-oauth-serving-cert\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:19:02.949031 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.949026 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:19:02.949220 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.949049 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-serving-cert\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:19:02.949220 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.949059 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-trusted-ca-bundle\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:19:02.949220 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.949067 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-console-oauth-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:19:02.949220 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:02.949076 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d6xj\" (UniqueName: \"kubernetes.io/projected/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1-kube-api-access-8d6xj\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:19:03.626993 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:03.626956 2569 generic.go:356] "Generic (PLEG): container finished" podID="955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" containerID="dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e" exitCode=0 Jul 02 14:19:03.627377 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:03.627014 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd878c44c-928lt" Jul 02 14:19:03.627377 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:03.627049 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd878c44c-928lt" event={"ID":"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1","Type":"ContainerDied","Data":"dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e"} Jul 02 14:19:03.627377 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:03.627111 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd878c44c-928lt" event={"ID":"955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1","Type":"ContainerDied","Data":"08602efe474c8fd4a6ec50f5740e245cd1baae97552c37fa3938a8c7a833e608"} Jul 02 14:19:03.627377 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:03.627133 2569 scope.go:117] "RemoveContainer" containerID="dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e" Jul 02 14:19:03.637077 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:03.637050 2569 scope.go:117] "RemoveContainer" containerID="dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e" Jul 02 14:19:03.637416 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:19:03.637386 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e\": container with ID starting with dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e not found: ID does not exist" containerID="dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e" Jul 02 14:19:03.637766 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:03.637427 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e"} err="failed to get container status \"dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e\": rpc error: code = NotFound desc = could not find container \"dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e\": container with ID starting with dbda68831eb61c7ee5ea6909ec9e706bd300d572726f5b7d65c29f1ea65e782e not found: ID does not exist" Jul 02 14:19:03.649692 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:03.649664 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd878c44c-928lt"] Jul 02 14:19:03.656097 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:03.656077 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cd878c44c-928lt"] Jul 02 14:19:04.597961 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:04.597927 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" path="/var/lib/kubelet/pods/955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1/volumes" Jul 02 14:19:22.849987 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:22.849944 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:19:22.864970 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:22.864941 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:19:23.710974 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:19:23.710944 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 14:20:36.542744 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:20:36.542711 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:20:36.546004 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:20:36.545980 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:20:36.546144 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:20:36.546026 2569 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 14:23:13.807340 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.807260 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-l2xnv"] Jul 02 14:23:13.807801 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.807636 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" containerName="console" Jul 02 14:23:13.807801 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.807654 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" containerName="console" Jul 02 14:23:13.807801 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.807734 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="955f3c95-7dd2-4e33-8da2-2feb4b2ab2b1" containerName="console" Jul 02 14:23:13.810560 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.810543 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:13.813469 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.813436 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Jul 02 14:23:13.813591 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.813528 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 02 14:23:13.813591 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.813539 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 02 14:23:13.814621 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.814606 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-b9bmd\"" Jul 02 14:23:13.820793 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.820769 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-l2xnv"] Jul 02 14:23:13.884163 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.884130 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79a2d70b-89ef-466e-8ffe-da4a14883ce4-cert\") pod \"odh-model-controller-84b54b7f97-l2xnv\" (UID: \"79a2d70b-89ef-466e-8ffe-da4a14883ce4\") " pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:13.884346 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.884192 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sdmw\" (UniqueName: \"kubernetes.io/projected/79a2d70b-89ef-466e-8ffe-da4a14883ce4-kube-api-access-9sdmw\") pod \"odh-model-controller-84b54b7f97-l2xnv\" (UID: \"79a2d70b-89ef-466e-8ffe-da4a14883ce4\") " pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:13.984705 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.984670 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79a2d70b-89ef-466e-8ffe-da4a14883ce4-cert\") pod \"odh-model-controller-84b54b7f97-l2xnv\" (UID: \"79a2d70b-89ef-466e-8ffe-da4a14883ce4\") " pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:13.984876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.984727 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sdmw\" (UniqueName: \"kubernetes.io/projected/79a2d70b-89ef-466e-8ffe-da4a14883ce4-kube-api-access-9sdmw\") pod \"odh-model-controller-84b54b7f97-l2xnv\" (UID: \"79a2d70b-89ef-466e-8ffe-da4a14883ce4\") " pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:13.984876 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:23:13.984819 2569 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 02 14:23:13.984965 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:23:13.984898 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79a2d70b-89ef-466e-8ffe-da4a14883ce4-cert podName:79a2d70b-89ef-466e-8ffe-da4a14883ce4 nodeName:}" failed. No retries permitted until 2026-07-02 14:23:14.484881231 +0000 UTC m=+458.359964665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/79a2d70b-89ef-466e-8ffe-da4a14883ce4-cert") pod "odh-model-controller-84b54b7f97-l2xnv" (UID: "79a2d70b-89ef-466e-8ffe-da4a14883ce4") : secret "odh-model-controller-webhook-cert" not found Jul 02 14:23:13.994832 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:13.994808 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sdmw\" (UniqueName: \"kubernetes.io/projected/79a2d70b-89ef-466e-8ffe-da4a14883ce4-kube-api-access-9sdmw\") pod \"odh-model-controller-84b54b7f97-l2xnv\" (UID: \"79a2d70b-89ef-466e-8ffe-da4a14883ce4\") " pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:14.489542 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:14.489511 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79a2d70b-89ef-466e-8ffe-da4a14883ce4-cert\") pod \"odh-model-controller-84b54b7f97-l2xnv\" (UID: \"79a2d70b-89ef-466e-8ffe-da4a14883ce4\") " pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:14.491844 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:14.491826 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79a2d70b-89ef-466e-8ffe-da4a14883ce4-cert\") pod \"odh-model-controller-84b54b7f97-l2xnv\" (UID: \"79a2d70b-89ef-466e-8ffe-da4a14883ce4\") " pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:14.721194 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:14.721155 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:14.848435 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:14.848408 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-l2xnv"] Jul 02 14:23:14.850937 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:23:14.850905 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a2d70b_89ef_466e_8ffe_da4a14883ce4.slice/crio-4737c64f6a2a557fe15f6c716726ffb0c1fcb4fb6c4cbe52f884cb08526c4c59 WatchSource:0}: Error finding container 4737c64f6a2a557fe15f6c716726ffb0c1fcb4fb6c4cbe52f884cb08526c4c59: Status 404 returned error can't find the container with id 4737c64f6a2a557fe15f6c716726ffb0c1fcb4fb6c4cbe52f884cb08526c4c59 Jul 02 14:23:14.852141 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:14.852121 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 14:23:15.487345 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:15.487314 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-l2xnv" event={"ID":"79a2d70b-89ef-466e-8ffe-da4a14883ce4","Type":"ContainerStarted","Data":"4737c64f6a2a557fe15f6c716726ffb0c1fcb4fb6c4cbe52f884cb08526c4c59"} Jul 02 14:23:18.501270 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:18.501226 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-l2xnv" event={"ID":"79a2d70b-89ef-466e-8ffe-da4a14883ce4","Type":"ContainerStarted","Data":"c79a56551e6fc811a39ca9ef2859a0f0ac6419b5ef0a07f7416bff92d1c512c5"} Jul 02 14:23:18.501746 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:18.501302 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:18.520309 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:18.520262 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-l2xnv" podStartSLOduration=2.269347903 podStartE2EDuration="5.520245406s" totalImagesPullingTime="3.250897503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:23:18.518798819 +0000 UTC m=+462.393882274" watchObservedRunningTime="2026-07-02 14:23:18.520245406 +0000 UTC m=+462.395328860" Jul 02 14:23:29.507619 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:29.507581 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-l2xnv" Jul 02 14:23:30.354948 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.354912 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-tjkzk"] Jul 02 14:23:30.357951 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.357934 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tjkzk" Jul 02 14:23:30.360807 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.360786 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-bmqvx\"" Jul 02 14:23:30.360901 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.360786 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 02 14:23:30.367007 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.366984 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-tjkzk"] Jul 02 14:23:30.423563 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.423535 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e100c45b-f8f8-4987-add2-ad05134f3b1d-model-cache\") pod \"s3-init-tjkzk\" (UID: \"e100c45b-f8f8-4987-add2-ad05134f3b1d\") " pod="kserve/s3-init-tjkzk" Jul 02 14:23:30.423715 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.423580 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-927p5\" (UniqueName: \"kubernetes.io/projected/e100c45b-f8f8-4987-add2-ad05134f3b1d-kube-api-access-927p5\") pod \"s3-init-tjkzk\" (UID: \"e100c45b-f8f8-4987-add2-ad05134f3b1d\") " pod="kserve/s3-init-tjkzk" Jul 02 14:23:30.524645 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.524606 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e100c45b-f8f8-4987-add2-ad05134f3b1d-model-cache\") pod \"s3-init-tjkzk\" (UID: \"e100c45b-f8f8-4987-add2-ad05134f3b1d\") " pod="kserve/s3-init-tjkzk" Jul 02 14:23:30.525115 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.524656 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-927p5\" (UniqueName: \"kubernetes.io/projected/e100c45b-f8f8-4987-add2-ad05134f3b1d-kube-api-access-927p5\") pod \"s3-init-tjkzk\" (UID: \"e100c45b-f8f8-4987-add2-ad05134f3b1d\") " pod="kserve/s3-init-tjkzk" Jul 02 14:23:30.525176 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.525109 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e100c45b-f8f8-4987-add2-ad05134f3b1d-model-cache\") pod \"s3-init-tjkzk\" (UID: \"e100c45b-f8f8-4987-add2-ad05134f3b1d\") " pod="kserve/s3-init-tjkzk" Jul 02 14:23:30.536299 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.536277 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-927p5\" (UniqueName: \"kubernetes.io/projected/e100c45b-f8f8-4987-add2-ad05134f3b1d-kube-api-access-927p5\") pod \"s3-init-tjkzk\" (UID: \"e100c45b-f8f8-4987-add2-ad05134f3b1d\") " pod="kserve/s3-init-tjkzk" Jul 02 14:23:30.686419 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.686373 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tjkzk" Jul 02 14:23:30.811974 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:30.811820 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-tjkzk"] Jul 02 14:23:30.814676 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:23:30.814643 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode100c45b_f8f8_4987_add2_ad05134f3b1d.slice/crio-a0977956b23321db9a154f8e2b02a63af78439fe1c7175057348ad24093a1c6e WatchSource:0}: Error finding container a0977956b23321db9a154f8e2b02a63af78439fe1c7175057348ad24093a1c6e: Status 404 returned error can't find the container with id a0977956b23321db9a154f8e2b02a63af78439fe1c7175057348ad24093a1c6e Jul 02 14:23:31.548696 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:31.548650 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tjkzk" event={"ID":"e100c45b-f8f8-4987-add2-ad05134f3b1d","Type":"ContainerStarted","Data":"a0977956b23321db9a154f8e2b02a63af78439fe1c7175057348ad24093a1c6e"} Jul 02 14:23:36.567811 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:36.567764 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tjkzk" event={"ID":"e100c45b-f8f8-4987-add2-ad05134f3b1d","Type":"ContainerStarted","Data":"ebc49fda5b0e29686c5e303824cb15af8b7cdd55fb9b2c40f021d348b312f3ee"} Jul 02 14:23:37.572733 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:37.572642 2569 generic.go:356] "Generic (PLEG): container finished" podID="e100c45b-f8f8-4987-add2-ad05134f3b1d" containerID="ebc49fda5b0e29686c5e303824cb15af8b7cdd55fb9b2c40f021d348b312f3ee" exitCode=0 Jul 02 14:23:37.572733 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:37.572687 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tjkzk" event={"ID":"e100c45b-f8f8-4987-add2-ad05134f3b1d","Type":"ContainerDied","Data":"ebc49fda5b0e29686c5e303824cb15af8b7cdd55fb9b2c40f021d348b312f3ee"} Jul 02 14:23:42.317823 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:42.317802 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 02 14:23:42.597545 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:42.597440 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tjkzk" event={"ID":"e100c45b-f8f8-4987-add2-ad05134f3b1d","Type":"ContainerStarted","Data":"f3c3f5e4976bb470e47e366f9a1fbf90f1d6f25fc289b2fb43cea2aaf0665076"} Jul 02 14:23:42.613175 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:23:42.613134 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-tjkzk" podStartSLOduration=1.565305255 podStartE2EDuration="12.61311941s" totalImagesPullingTime="9.496685037s" totalInitContainerRuntime="1.551129118s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:23:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:23:42.611486062 +0000 UTC m=+486.486569534" watchObservedRunningTime="2026-07-02 14:23:42.61311941 +0000 UTC m=+486.488202865" Jul 02 14:24:47.818919 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:47.818883 2569 generic.go:356] "Generic (PLEG): container finished" podID="e100c45b-f8f8-4987-add2-ad05134f3b1d" containerID="f3c3f5e4976bb470e47e366f9a1fbf90f1d6f25fc289b2fb43cea2aaf0665076" exitCode=0 Jul 02 14:24:47.819329 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:47.818940 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tjkzk" event={"ID":"e100c45b-f8f8-4987-add2-ad05134f3b1d","Type":"ContainerDied","Data":"f3c3f5e4976bb470e47e366f9a1fbf90f1d6f25fc289b2fb43cea2aaf0665076"} Jul 02 14:24:48.946951 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:48.946925 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tjkzk" Jul 02 14:24:49.055097 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:49.055062 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-927p5\" (UniqueName: \"kubernetes.io/projected/e100c45b-f8f8-4987-add2-ad05134f3b1d-kube-api-access-927p5\") pod \"e100c45b-f8f8-4987-add2-ad05134f3b1d\" (UID: \"e100c45b-f8f8-4987-add2-ad05134f3b1d\") " Jul 02 14:24:49.055097 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:49.055100 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e100c45b-f8f8-4987-add2-ad05134f3b1d-model-cache\") pod \"e100c45b-f8f8-4987-add2-ad05134f3b1d\" (UID: \"e100c45b-f8f8-4987-add2-ad05134f3b1d\") " Jul 02 14:24:49.057257 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:49.057224 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e100c45b-f8f8-4987-add2-ad05134f3b1d-kube-api-access-927p5" (OuterVolumeSpecName: "kube-api-access-927p5") pod "e100c45b-f8f8-4987-add2-ad05134f3b1d" (UID: "e100c45b-f8f8-4987-add2-ad05134f3b1d"). InnerVolumeSpecName "kube-api-access-927p5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:24:49.074916 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:49.074860 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e100c45b-f8f8-4987-add2-ad05134f3b1d-model-cache" (OuterVolumeSpecName: "model-cache") pod "e100c45b-f8f8-4987-add2-ad05134f3b1d" (UID: "e100c45b-f8f8-4987-add2-ad05134f3b1d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:24:49.156532 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:49.156501 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-927p5\" (UniqueName: \"kubernetes.io/projected/e100c45b-f8f8-4987-add2-ad05134f3b1d-kube-api-access-927p5\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:24:49.156532 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:49.156528 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e100c45b-f8f8-4987-add2-ad05134f3b1d-model-cache\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:24:49.828032 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:49.827989 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tjkzk" event={"ID":"e100c45b-f8f8-4987-add2-ad05134f3b1d","Type":"ContainerDied","Data":"a0977956b23321db9a154f8e2b02a63af78439fe1c7175057348ad24093a1c6e"} Jul 02 14:24:49.828032 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:49.828021 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tjkzk" Jul 02 14:24:49.828032 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:24:49.828033 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0977956b23321db9a154f8e2b02a63af78439fe1c7175057348ad24093a1c6e" Jul 02 14:25:36.565194 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:25:36.565167 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:25:36.566516 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:25:36.566495 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:26:57.297104 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.297066 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq"] Jul 02 14:26:57.297534 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.297362 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e100c45b-f8f8-4987-add2-ad05134f3b1d" containerName="download-hf-model" Jul 02 14:26:57.297534 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.297373 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e100c45b-f8f8-4987-add2-ad05134f3b1d" containerName="download-hf-model" Jul 02 14:26:57.297534 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.297390 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e100c45b-f8f8-4987-add2-ad05134f3b1d" containerName="s3-init" Jul 02 14:26:57.297534 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.297396 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e100c45b-f8f8-4987-add2-ad05134f3b1d" containerName="s3-init" Jul 02 14:26:57.297534 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.297462 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e100c45b-f8f8-4987-add2-ad05134f3b1d" containerName="s3-init" Jul 02 14:26:57.300393 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.300375 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.303138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.303112 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-z6qw4\"" Jul 02 14:26:57.303253 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.303113 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-042a3-kube-rbac-proxy-sar-config\"" Jul 02 14:26:57.303253 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.303137 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 02 14:26:57.303253 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.303169 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-042a3-predictor-serving-cert\"" Jul 02 14:26:57.303253 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.303184 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 02 14:26:57.303253 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.303192 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 02 14:26:57.303253 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.303140 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 02 14:26:57.311061 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.311039 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq"] Jul 02 14:26:57.359026 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.358993 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1d127a37-158a-4a19-afd6-d536738f4108-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.359196 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.359049 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1d127a37-158a-4a19-afd6-d536738f4108-proxy-tls\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.359252 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.359181 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmpwh\" (UniqueName: \"kubernetes.io/projected/1d127a37-158a-4a19-afd6-d536738f4108-kube-api-access-fmpwh\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.359252 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.359238 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.359376 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.359283 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-cabundle-cert\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.460351 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.460318 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmpwh\" (UniqueName: \"kubernetes.io/projected/1d127a37-158a-4a19-afd6-d536738f4108-kube-api-access-fmpwh\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.460351 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.460360 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.460647 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.460384 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-cabundle-cert\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.460647 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.460407 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1d127a37-158a-4a19-afd6-d536738f4108-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.460647 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.460432 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1d127a37-158a-4a19-afd6-d536738f4108-proxy-tls\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.460933 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.460907 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1d127a37-158a-4a19-afd6-d536738f4108-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.461105 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.461082 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-cabundle-cert\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.461216 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.461197 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.462956 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.462940 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1d127a37-158a-4a19-afd6-d536738f4108-proxy-tls\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.470436 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.470415 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmpwh\" (UniqueName: \"kubernetes.io/projected/1d127a37-158a-4a19-afd6-d536738f4108-kube-api-access-fmpwh\") pod \"isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.611281 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.611190 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:26:57.741320 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:57.741287 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq"] Jul 02 14:26:57.744533 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:26:57.744503 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d127a37_158a_4a19_afd6_d536738f4108.slice/crio-98c69c08f5145a31f330170522cf9454bfa7715da8b21a0eb02605b5be4b03fb WatchSource:0}: Error finding container 98c69c08f5145a31f330170522cf9454bfa7715da8b21a0eb02605b5be4b03fb: Status 404 returned error can't find the container with id 98c69c08f5145a31f330170522cf9454bfa7715da8b21a0eb02605b5be4b03fb Jul 02 14:26:58.274212 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:58.274168 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" event={"ID":"1d127a37-158a-4a19-afd6-d536738f4108","Type":"ContainerStarted","Data":"fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f"} Jul 02 14:26:58.274391 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:26:58.274234 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" event={"ID":"1d127a37-158a-4a19-afd6-d536738f4108","Type":"ContainerStarted","Data":"98c69c08f5145a31f330170522cf9454bfa7715da8b21a0eb02605b5be4b03fb"} Jul 02 14:27:01.287133 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:01.287099 2569 generic.go:356] "Generic (PLEG): container finished" podID="1d127a37-158a-4a19-afd6-d536738f4108" containerID="fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f" exitCode=0 Jul 02 14:27:01.287541 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:01.287176 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" event={"ID":"1d127a37-158a-4a19-afd6-d536738f4108","Type":"ContainerDied","Data":"fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f"} Jul 02 14:27:24.379939 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:24.379905 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" event={"ID":"1d127a37-158a-4a19-afd6-d536738f4108","Type":"ContainerStarted","Data":"8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273"} Jul 02 14:27:27.394555 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:27.394518 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" event={"ID":"1d127a37-158a-4a19-afd6-d536738f4108","Type":"ContainerStarted","Data":"c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262"} Jul 02 14:27:27.395107 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:27.394642 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:27:27.416795 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:27.416748 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podStartSLOduration=1.83509293 podStartE2EDuration="30.416734699s" totalImagesPullingTime="25.138869882s" totalInitContainerRuntime="3.442771887s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:26:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:27:27.414780371 +0000 UTC m=+711.289863827" watchObservedRunningTime="2026-07-02 14:27:27.416734699 +0000 UTC m=+711.291818154" Jul 02 14:27:28.398711 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:28.398670 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:27:28.399969 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:28.399946 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 02 14:27:29.402474 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:29.402423 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 02 14:27:34.406341 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:34.406312 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:27:34.406835 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:34.406812 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 02 14:27:44.406817 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:44.406771 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 02 14:27:54.407542 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:27:54.407496 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 02 14:28:04.407253 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:04.407209 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 02 14:28:14.407467 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:14.407409 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 02 14:28:24.407437 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:24.407407 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:28:47.610832 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.610799 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg"] Jul 02 14:28:47.616189 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.616161 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.619001 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.618976 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-hpa-130af-predictor-serving-cert\"" Jul 02 14:28:47.619138 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.619081 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-hpa-130af-kube-rbac-proxy-sar-config\"" Jul 02 14:28:47.626808 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.626784 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg"] Jul 02 14:28:47.641547 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.641517 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq"] Jul 02 14:28:47.641754 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.641728 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" containerID="cri-o://8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273" gracePeriod=30 Jul 02 14:28:47.641815 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.641751 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kube-rbac-proxy" containerID="cri-o://c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262" gracePeriod=30 Jul 02 14:28:47.671172 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.671142 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvtq9\" (UniqueName: \"kubernetes.io/projected/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-api-access-qvtq9\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.671313 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.671185 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.671313 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.671258 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.671313 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.671299 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.671525 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.671331 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.772010 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.771979 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvtq9\" (UniqueName: \"kubernetes.io/projected/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-api-access-qvtq9\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.772010 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.772012 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.772232 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.772210 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.772289 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.772275 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.772341 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.772321 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.772704 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.772676 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.772959 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.772936 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.773045 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.772980 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.774501 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.774480 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.780060 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.780034 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvtq9\" (UniqueName: \"kubernetes.io/projected/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-api-access-qvtq9\") pod \"isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:47.927899 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:47.927859 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:48.056049 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:48.056011 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg"] Jul 02 14:28:48.058008 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:28:48.057980 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b57e6f_ffe2_4605_b2b0_365569b1bfd2.slice/crio-d312598b6ad5e91457369372bcd88db83ff34b8fd0d0b865ece60cfbb7d08cbf WatchSource:0}: Error finding container d312598b6ad5e91457369372bcd88db83ff34b8fd0d0b865ece60cfbb7d08cbf: Status 404 returned error can't find the container with id d312598b6ad5e91457369372bcd88db83ff34b8fd0d0b865ece60cfbb7d08cbf Jul 02 14:28:48.697789 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:48.697759 2569 generic.go:356] "Generic (PLEG): container finished" podID="1d127a37-158a-4a19-afd6-d536738f4108" containerID="c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262" exitCode=2 Jul 02 14:28:48.698199 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:48.697820 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" event={"ID":"1d127a37-158a-4a19-afd6-d536738f4108","Type":"ContainerDied","Data":"c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262"} Jul 02 14:28:48.699501 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:48.699473 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" event={"ID":"04b57e6f-ffe2-4605-b2b0-365569b1bfd2","Type":"ContainerStarted","Data":"804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f"} Jul 02 14:28:48.699630 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:48.699525 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" event={"ID":"04b57e6f-ffe2-4605-b2b0-365569b1bfd2","Type":"ContainerStarted","Data":"d312598b6ad5e91457369372bcd88db83ff34b8fd0d0b865ece60cfbb7d08cbf"} Jul 02 14:28:49.403710 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:49.403665 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.22:8643/healthz\": dial tcp 10.133.0.22:8643: connect: connection refused" Jul 02 14:28:51.182412 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.182387 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:28:51.198275 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.198247 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-kube-rbac-proxy-sar-config\") pod \"1d127a37-158a-4a19-afd6-d536738f4108\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " Jul 02 14:28:51.198402 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.198281 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1d127a37-158a-4a19-afd6-d536738f4108-proxy-tls\") pod \"1d127a37-158a-4a19-afd6-d536738f4108\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " Jul 02 14:28:51.198402 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.198319 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1d127a37-158a-4a19-afd6-d536738f4108-kserve-provision-location\") pod \"1d127a37-158a-4a19-afd6-d536738f4108\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " Jul 02 14:28:51.198402 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.198394 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmpwh\" (UniqueName: \"kubernetes.io/projected/1d127a37-158a-4a19-afd6-d536738f4108-kube-api-access-fmpwh\") pod \"1d127a37-158a-4a19-afd6-d536738f4108\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " Jul 02 14:28:51.198562 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.198475 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-cabundle-cert\") pod \"1d127a37-158a-4a19-afd6-d536738f4108\" (UID: \"1d127a37-158a-4a19-afd6-d536738f4108\") " Jul 02 14:28:51.198769 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.198687 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1d127a37-158a-4a19-afd6-d536738f4108" (UID: "1d127a37-158a-4a19-afd6-d536738f4108"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:28:51.198873 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.198848 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d127a37-158a-4a19-afd6-d536738f4108-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1d127a37-158a-4a19-afd6-d536738f4108" (UID: "1d127a37-158a-4a19-afd6-d536738f4108"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:28:51.198912 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.198863 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1d127a37-158a-4a19-afd6-d536738f4108" (UID: "1d127a37-158a-4a19-afd6-d536738f4108"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:28:51.200650 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.200626 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d127a37-158a-4a19-afd6-d536738f4108-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1d127a37-158a-4a19-afd6-d536738f4108" (UID: "1d127a37-158a-4a19-afd6-d536738f4108"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:28:51.200650 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.200639 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d127a37-158a-4a19-afd6-d536738f4108-kube-api-access-fmpwh" (OuterVolumeSpecName: "kube-api-access-fmpwh") pod "1d127a37-158a-4a19-afd6-d536738f4108" (UID: "1d127a37-158a-4a19-afd6-d536738f4108"). InnerVolumeSpecName "kube-api-access-fmpwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:28:51.300076 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.299995 2569 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-cabundle-cert\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:28:51.300076 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.300024 2569 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1d127a37-158a-4a19-afd6-d536738f4108-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:28:51.300076 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.300037 2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1d127a37-158a-4a19-afd6-d536738f4108-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:28:51.300076 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.300045 2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1d127a37-158a-4a19-afd6-d536738f4108-kserve-provision-location\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:28:51.300076 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.300053 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmpwh\" (UniqueName: \"kubernetes.io/projected/1d127a37-158a-4a19-afd6-d536738f4108-kube-api-access-fmpwh\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:28:51.718075 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.718039 2569 generic.go:356] "Generic (PLEG): container finished" podID="1d127a37-158a-4a19-afd6-d536738f4108" containerID="8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273" exitCode=0 Jul 02 14:28:51.718237 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.718083 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" event={"ID":"1d127a37-158a-4a19-afd6-d536738f4108","Type":"ContainerDied","Data":"8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273"} Jul 02 14:28:51.718237 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.718105 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" Jul 02 14:28:51.718237 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.718143 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq" event={"ID":"1d127a37-158a-4a19-afd6-d536738f4108","Type":"ContainerDied","Data":"98c69c08f5145a31f330170522cf9454bfa7715da8b21a0eb02605b5be4b03fb"} Jul 02 14:28:51.718237 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.718180 2569 scope.go:117] "RemoveContainer" containerID="c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262" Jul 02 14:28:51.726473 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.726433 2569 scope.go:117] "RemoveContainer" containerID="8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273" Jul 02 14:28:51.733334 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.733318 2569 scope.go:117] "RemoveContainer" containerID="fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f" Jul 02 14:28:51.740287 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.740270 2569 scope.go:117] "RemoveContainer" containerID="c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262" Jul 02 14:28:51.740564 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:28:51.740545 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262\": container with ID starting with c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262 not found: ID does not exist" containerID="c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262" Jul 02 14:28:51.740622 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.740573 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262"} err="failed to get container status \"c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262\": rpc error: code = NotFound desc = could not find container \"c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262\": container with ID starting with c8420cfc47580473c00f97b1f09947949d1dba2852d942c4458feefef280e262 not found: ID does not exist" Jul 02 14:28:51.740622 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.740594 2569 scope.go:117] "RemoveContainer" containerID="8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273" Jul 02 14:28:51.740851 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:28:51.740836 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273\": container with ID starting with 8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273 not found: ID does not exist" containerID="8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273" Jul 02 14:28:51.740899 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.740854 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273"} err="failed to get container status \"8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273\": rpc error: code = NotFound desc = could not find container \"8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273\": container with ID starting with 8d91b4105a0be75122730c35ab36707de582d9c2e8ce461e9ab1db42b2429273 not found: ID does not exist" Jul 02 14:28:51.740899 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.740867 2569 scope.go:117] "RemoveContainer" containerID="fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f" Jul 02 14:28:51.741058 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:28:51.741042 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f\": container with ID starting with fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f not found: ID does not exist" containerID="fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f" Jul 02 14:28:51.741096 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.741062 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f"} err="failed to get container status \"fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f\": rpc error: code = NotFound desc = could not find container \"fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f\": container with ID starting with fb88f0b250dec1176a84cd36649b888c3502d127ba77a738d4bd31ee355e859f not found: ID does not exist" Jul 02 14:28:51.744155 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.744135 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq"] Jul 02 14:28:51.748182 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:51.748162 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-042a3-predictor-77d54fc7f7-xb6jq"] Jul 02 14:28:52.597576 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:52.597544 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d127a37-158a-4a19-afd6-d536738f4108" path="/var/lib/kubelet/pods/1d127a37-158a-4a19-afd6-d536738f4108/volumes" Jul 02 14:28:52.724654 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:52.724623 2569 generic.go:356] "Generic (PLEG): container finished" podID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerID="804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f" exitCode=0 Jul 02 14:28:52.724811 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:52.724661 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" event={"ID":"04b57e6f-ffe2-4605-b2b0-365569b1bfd2","Type":"ContainerDied","Data":"804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f"} Jul 02 14:28:53.729851 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:53.729815 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" event={"ID":"04b57e6f-ffe2-4605-b2b0-365569b1bfd2","Type":"ContainerStarted","Data":"35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344"} Jul 02 14:28:53.730200 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:53.729878 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" event={"ID":"04b57e6f-ffe2-4605-b2b0-365569b1bfd2","Type":"ContainerStarted","Data":"590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9"} Jul 02 14:28:53.730200 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:53.730059 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:53.751411 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:53.751357 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podStartSLOduration=3.067110439 podStartE2EDuration="6.751342652s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.684232213s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:28:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:28:53.749726258 +0000 UTC m=+797.624809732" watchObservedRunningTime="2026-07-02 14:28:53.751342652 +0000 UTC m=+797.626426178" Jul 02 14:28:54.733946 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:54.733919 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:28:54.734954 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:54.734926 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 02 14:28:55.737338 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:28:55.737289 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 02 14:29:00.742050 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:29:00.741978 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:29:00.742585 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:29:00.742560 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 02 14:29:10.743069 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:29:10.743031 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 02 14:29:20.743179 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:29:20.743140 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 02 14:29:30.743498 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:29:30.743441 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 02 14:29:40.742901 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:29:40.742864 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 02 14:29:50.743296 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:29:50.743262 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 02 14:30:00.743503 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:00.743470 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:30:36.587592 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:36.587523 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:30:36.588127 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:36.587589 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:30:37.927636 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:37.927606 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg"] Jul 02 14:30:37.928037 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:37.927827 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" containerID="cri-o://590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9" gracePeriod=30 Jul 02 14:30:37.928037 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:37.927880 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kube-rbac-proxy" containerID="cri-o://35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344" gracePeriod=30 Jul 02 14:30:38.121136 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:38.121095 2569 generic.go:356] "Generic (PLEG): container finished" podID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerID="35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344" exitCode=2 Jul 02 14:30:38.121287 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:38.121145 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" event={"ID":"04b57e6f-ffe2-4605-b2b0-365569b1bfd2","Type":"ContainerDied","Data":"35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344"} Jul 02 14:30:40.738131 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:40.738089 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.23:8643/healthz\": dial tcp 10.133.0.23:8643: connect: connection refused" Jul 02 14:30:40.742605 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:40.742579 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 02 14:30:41.562289 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.562266 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:30:41.598418 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.598389 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvtq9\" (UniqueName: \"kubernetes.io/projected/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-api-access-qvtq9\") pod \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " Jul 02 14:30:41.598606 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.598437 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-proxy-tls\") pod \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " Jul 02 14:30:41.598606 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.598490 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-cabundle-cert\") pod \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " Jul 02 14:30:41.598606 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.598526 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kserve-provision-location\") pod \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " Jul 02 14:30:41.598606 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.598581 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-rbac-proxy-sar-config\") pod \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\" (UID: \"04b57e6f-ffe2-4605-b2b0-365569b1bfd2\") " Jul 02 14:30:41.598935 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.598905 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "04b57e6f-ffe2-4605-b2b0-365569b1bfd2" (UID: "04b57e6f-ffe2-4605-b2b0-365569b1bfd2"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:30:41.599058 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.598928 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "04b57e6f-ffe2-4605-b2b0-365569b1bfd2" (UID: "04b57e6f-ffe2-4605-b2b0-365569b1bfd2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:30:41.599118 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.599075 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "04b57e6f-ffe2-4605-b2b0-365569b1bfd2" (UID: "04b57e6f-ffe2-4605-b2b0-365569b1bfd2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:30:41.601044 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.601017 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "04b57e6f-ffe2-4605-b2b0-365569b1bfd2" (UID: "04b57e6f-ffe2-4605-b2b0-365569b1bfd2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:30:41.601104 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.601035 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-api-access-qvtq9" (OuterVolumeSpecName: "kube-api-access-qvtq9") pod "04b57e6f-ffe2-4605-b2b0-365569b1bfd2" (UID: "04b57e6f-ffe2-4605-b2b0-365569b1bfd2"). InnerVolumeSpecName "kube-api-access-qvtq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:30:41.699197 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.699111 2569 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:30:41.699197 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.699143 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvtq9\" (UniqueName: \"kubernetes.io/projected/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kube-api-access-qvtq9\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:30:41.699197 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.699153 2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:30:41.699197 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.699162 2569 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-cabundle-cert\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:30:41.699197 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:41.699170 2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b57e6f-ffe2-4605-b2b0-365569b1bfd2-kserve-provision-location\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:30:42.137355 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.137325 2569 generic.go:356] "Generic (PLEG): container finished" podID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerID="590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9" exitCode=0 Jul 02 14:30:42.137792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.137361 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" event={"ID":"04b57e6f-ffe2-4605-b2b0-365569b1bfd2","Type":"ContainerDied","Data":"590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9"} Jul 02 14:30:42.137792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.137381 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" Jul 02 14:30:42.137792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.137409 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg" event={"ID":"04b57e6f-ffe2-4605-b2b0-365569b1bfd2","Type":"ContainerDied","Data":"d312598b6ad5e91457369372bcd88db83ff34b8fd0d0b865ece60cfbb7d08cbf"} Jul 02 14:30:42.137792 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.137435 2569 scope.go:117] "RemoveContainer" containerID="35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344" Jul 02 14:30:42.147684 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.147666 2569 scope.go:117] "RemoveContainer" containerID="590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9" Jul 02 14:30:42.155257 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.155239 2569 scope.go:117] "RemoveContainer" containerID="804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f" Jul 02 14:30:42.161898 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.161870 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg"] Jul 02 14:30:42.163232 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.163212 2569 scope.go:117] "RemoveContainer" containerID="35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344" Jul 02 14:30:42.163660 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:30:42.163633 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344\": container with ID starting with 35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344 not found: ID does not exist" containerID="35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344" Jul 02 14:30:42.163762 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.163671 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344"} err="failed to get container status \"35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344\": rpc error: code = NotFound desc = could not find container \"35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344\": container with ID starting with 35d5df68f2eb307d65028b92ae98bdd78c7e103c694ac28365b4047f3b593344 not found: ID does not exist" Jul 02 14:30:42.163762 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.163694 2569 scope.go:117] "RemoveContainer" containerID="590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9" Jul 02 14:30:42.164130 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:30:42.164104 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9\": container with ID starting with 590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9 not found: ID does not exist" containerID="590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9" Jul 02 14:30:42.164225 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.164139 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9"} err="failed to get container status \"590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9\": rpc error: code = NotFound desc = could not find container \"590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9\": container with ID starting with 590aed7609d036e38bc0cc1f3fe23117cc6b4406cce82ff42fcb6dd75f81f7e9 not found: ID does not exist" Jul 02 14:30:42.164225 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.164183 2569 scope.go:117] "RemoveContainer" containerID="804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f" Jul 02 14:30:42.164642 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:30:42.164620 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f\": container with ID starting with 804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f not found: ID does not exist" containerID="804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f" Jul 02 14:30:42.164724 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.164650 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f"} err="failed to get container status \"804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f\": rpc error: code = NotFound desc = could not find container \"804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f\": container with ID starting with 804d1bc2af4930fcdf40ef385631e864de540b4f004e637d76466124bf9b914f not found: ID does not exist" Jul 02 14:30:42.165588 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.165569 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-130af-predictor-86bb7cf7bf-l7dvg"] Jul 02 14:30:42.597725 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:30:42.597638 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" path="/var/lib/kubelet/pods/04b57e6f-ffe2-4605-b2b0-365569b1bfd2/volumes" Jul 02 14:35:36.609719 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:35:36.609681 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:35:36.610427 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:35:36.610407 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:40:25.928154 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928121 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k7pk7/must-gather-v4lqp"] Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928386 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kube-rbac-proxy" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928396 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kube-rbac-proxy" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928412 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kube-rbac-proxy" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928418 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kube-rbac-proxy" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928426 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928431 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928440 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="storage-initializer" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928463 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="storage-initializer" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928476 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="storage-initializer" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928482 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="storage-initializer" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928488 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928493 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928531 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kube-rbac-proxy" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928539 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kserve-container" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928544 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b57e6f-ffe2-4605-b2b0-365569b1bfd2" containerName="kube-rbac-proxy" Jul 02 14:40:25.928658 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.928552 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d127a37-158a-4a19-afd6-d536738f4108" containerName="kserve-container" Jul 02 14:40:25.931160 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.931144 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:40:25.933846 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.933825 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k7pk7\"/\"kube-root-ca.crt\"" Jul 02 14:40:25.933984 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.933825 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k7pk7\"/\"default-dockercfg-2ndk9\"" Jul 02 14:40:25.933984 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.933958 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k7pk7\"/\"openshift-service-ca.crt\"" Jul 02 14:40:25.940131 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:25.940107 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k7pk7/must-gather-v4lqp"] Jul 02 14:40:26.059720 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:26.059682 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/18649e26-dba6-49f4-a32e-b9cf29365168-must-gather-output\") pod \"must-gather-v4lqp\" (UID: \"18649e26-dba6-49f4-a32e-b9cf29365168\") " pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:40:26.059907 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:26.059746 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz9hk\" (UniqueName: \"kubernetes.io/projected/18649e26-dba6-49f4-a32e-b9cf29365168-kube-api-access-vz9hk\") pod \"must-gather-v4lqp\" (UID: \"18649e26-dba6-49f4-a32e-b9cf29365168\") " pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:40:26.160968 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:26.160922 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz9hk\" (UniqueName: \"kubernetes.io/projected/18649e26-dba6-49f4-a32e-b9cf29365168-kube-api-access-vz9hk\") pod \"must-gather-v4lqp\" (UID: \"18649e26-dba6-49f4-a32e-b9cf29365168\") " pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:40:26.161109 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:26.161080 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/18649e26-dba6-49f4-a32e-b9cf29365168-must-gather-output\") pod \"must-gather-v4lqp\" (UID: \"18649e26-dba6-49f4-a32e-b9cf29365168\") " pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:40:26.161413 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:26.161390 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/18649e26-dba6-49f4-a32e-b9cf29365168-must-gather-output\") pod \"must-gather-v4lqp\" (UID: \"18649e26-dba6-49f4-a32e-b9cf29365168\") " pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:40:26.169770 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:26.169744 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz9hk\" (UniqueName: \"kubernetes.io/projected/18649e26-dba6-49f4-a32e-b9cf29365168-kube-api-access-vz9hk\") pod \"must-gather-v4lqp\" (UID: \"18649e26-dba6-49f4-a32e-b9cf29365168\") " pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:40:26.256257 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:26.256183 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:40:26.383635 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:26.383606 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k7pk7/must-gather-v4lqp"] Jul 02 14:40:26.384237 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:40:26.384213 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18649e26_dba6_49f4_a32e_b9cf29365168.slice/crio-25dd0df76ec18b6bc8b5c93f7ea01ea85c3b44f51418b958b896d4316729ab29 WatchSource:0}: Error finding container 25dd0df76ec18b6bc8b5c93f7ea01ea85c3b44f51418b958b896d4316729ab29: Status 404 returned error can't find the container with id 25dd0df76ec18b6bc8b5c93f7ea01ea85c3b44f51418b958b896d4316729ab29 Jul 02 14:40:26.386103 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:26.386083 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 14:40:27.202284 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:27.202250 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" event={"ID":"18649e26-dba6-49f4-a32e-b9cf29365168","Type":"ContainerStarted","Data":"25dd0df76ec18b6bc8b5c93f7ea01ea85c3b44f51418b958b896d4316729ab29"} Jul 02 14:40:33.232669 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:33.232632 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" event={"ID":"18649e26-dba6-49f4-a32e-b9cf29365168","Type":"ContainerStarted","Data":"3e90afdf00e1faa2b0a6ba62641e318a75db37e59022a52091515916e0fd3439"} Jul 02 14:40:33.232669 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:33.232673 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" event={"ID":"18649e26-dba6-49f4-a32e-b9cf29365168","Type":"ContainerStarted","Data":"d704166ce68cd4ed20e07addd87776f1b2b572c67e2b928e22a30262bac35d51"} Jul 02 14:40:33.252663 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:33.252572 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" podStartSLOduration=1.9027598019999998 podStartE2EDuration="8.252556826s" totalImagesPullingTime="6.349797024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:40:33.250155234 +0000 UTC m=+1497.125238690" watchObservedRunningTime="2026-07-02 14:40:33.252556826 +0000 UTC m=+1497.127640294" Jul 02 14:40:36.634593 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:36.634565 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:40:36.636278 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:36.636257 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:40:54.317384 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:54.317345 2569 generic.go:356] "Generic (PLEG): container finished" podID="18649e26-dba6-49f4-a32e-b9cf29365168" containerID="d704166ce68cd4ed20e07addd87776f1b2b572c67e2b928e22a30262bac35d51" exitCode=0 Jul 02 14:40:54.317872 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:54.317419 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" event={"ID":"18649e26-dba6-49f4-a32e-b9cf29365168","Type":"ContainerDied","Data":"d704166ce68cd4ed20e07addd87776f1b2b572c67e2b928e22a30262bac35d51"} Jul 02 14:40:54.317872 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:54.317751 2569 scope.go:117] "RemoveContainer" containerID="d704166ce68cd4ed20e07addd87776f1b2b572c67e2b928e22a30262bac35d51" Jul 02 14:40:54.703239 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:54.703202 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k7pk7_must-gather-v4lqp_18649e26-dba6-49f4-a32e-b9cf29365168/gather/0.log" Jul 02 14:40:55.320111 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.320079 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-52qrt/must-gather-f8w8f"] Jul 02 14:40:55.322382 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.322362 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52qrt/must-gather-f8w8f" Jul 02 14:40:55.325155 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.325130 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-52qrt\"/\"openshift-service-ca.crt\"" Jul 02 14:40:55.326508 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.326491 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-52qrt\"/\"default-dockercfg-mpkrj\"" Jul 02 14:40:55.326607 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.326492 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-52qrt\"/\"kube-root-ca.crt\"" Jul 02 14:40:55.332544 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.332522 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52qrt/must-gather-f8w8f"] Jul 02 14:40:55.380188 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.380153 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcwgc\" (UniqueName: \"kubernetes.io/projected/ba9564de-aa0e-42bd-bf82-528d84c5926f-kube-api-access-mcwgc\") pod \"must-gather-f8w8f\" (UID: \"ba9564de-aa0e-42bd-bf82-528d84c5926f\") " pod="openshift-must-gather-52qrt/must-gather-f8w8f" Jul 02 14:40:55.380346 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.380204 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ba9564de-aa0e-42bd-bf82-528d84c5926f-must-gather-output\") pod \"must-gather-f8w8f\" (UID: \"ba9564de-aa0e-42bd-bf82-528d84c5926f\") " pod="openshift-must-gather-52qrt/must-gather-f8w8f" Jul 02 14:40:55.481400 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.481365 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcwgc\" (UniqueName: \"kubernetes.io/projected/ba9564de-aa0e-42bd-bf82-528d84c5926f-kube-api-access-mcwgc\") pod \"must-gather-f8w8f\" (UID: \"ba9564de-aa0e-42bd-bf82-528d84c5926f\") " pod="openshift-must-gather-52qrt/must-gather-f8w8f" Jul 02 14:40:55.481567 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.481417 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ba9564de-aa0e-42bd-bf82-528d84c5926f-must-gather-output\") pod \"must-gather-f8w8f\" (UID: \"ba9564de-aa0e-42bd-bf82-528d84c5926f\") " pod="openshift-must-gather-52qrt/must-gather-f8w8f" Jul 02 14:40:55.481778 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.481761 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ba9564de-aa0e-42bd-bf82-528d84c5926f-must-gather-output\") pod \"must-gather-f8w8f\" (UID: \"ba9564de-aa0e-42bd-bf82-528d84c5926f\") " pod="openshift-must-gather-52qrt/must-gather-f8w8f" Jul 02 14:40:55.490429 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.490403 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcwgc\" (UniqueName: \"kubernetes.io/projected/ba9564de-aa0e-42bd-bf82-528d84c5926f-kube-api-access-mcwgc\") pod \"must-gather-f8w8f\" (UID: \"ba9564de-aa0e-42bd-bf82-528d84c5926f\") " pod="openshift-must-gather-52qrt/must-gather-f8w8f" Jul 02 14:40:55.631134 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.631106 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52qrt/must-gather-f8w8f" Jul 02 14:40:55.759610 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:55.759579 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52qrt/must-gather-f8w8f"] Jul 02 14:40:55.762550 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:40:55.762518 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba9564de_aa0e_42bd_bf82_528d84c5926f.slice/crio-743cf35c203f4aa94c1afd5030e670cb57592b21a7c216bdd33f03c1753b8706 WatchSource:0}: Error finding container 743cf35c203f4aa94c1afd5030e670cb57592b21a7c216bdd33f03c1753b8706: Status 404 returned error can't find the container with id 743cf35c203f4aa94c1afd5030e670cb57592b21a7c216bdd33f03c1753b8706 Jul 02 14:40:56.326300 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:56.326266 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52qrt/must-gather-f8w8f" event={"ID":"ba9564de-aa0e-42bd-bf82-528d84c5926f","Type":"ContainerStarted","Data":"743cf35c203f4aa94c1afd5030e670cb57592b21a7c216bdd33f03c1753b8706"} Jul 02 14:40:57.333496 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:57.333431 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52qrt/must-gather-f8w8f" event={"ID":"ba9564de-aa0e-42bd-bf82-528d84c5926f","Type":"ContainerStarted","Data":"5ff6d6678296a4efd5a67c94c3067a74c5fa6828c117406010d65a7a4274b3ac"} Jul 02 14:40:57.333979 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:57.333507 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52qrt/must-gather-f8w8f" event={"ID":"ba9564de-aa0e-42bd-bf82-528d84c5926f","Type":"ContainerStarted","Data":"cfa3d238389189459c423a2b7b7487bcf9111756a4abfacd4ad29e668e2bbe00"} Jul 02 14:40:57.353018 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:57.352948 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-52qrt/must-gather-f8w8f" podStartSLOduration=1.429220776 podStartE2EDuration="2.35292694s" totalImagesPullingTime="923.706164ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:40:57.350536133 +0000 UTC m=+1521.225619590" watchObservedRunningTime="2026-07-02 14:40:57.35292694 +0000 UTC m=+1521.228010396" Jul 02 14:40:58.149483 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:58.149437 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-df6mz_5b33b423-4de4-41fa-85de-d5ae343c63aa/global-pull-secret-syncer/0.log" Jul 02 14:40:58.282974 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:58.282928 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dcshg_2833a111-c1fc-4670-92e5-9dfacb75df80/konnectivity-agent/0.log" Jul 02 14:40:58.396116 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:40:58.396083 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-134.ec2.internal_15d4eff56ed4ced4d7d17ab47ab02ee1/haproxy/0.log" Jul 02 14:41:00.128171 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.128085 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-k7pk7/must-gather-v4lqp"] Jul 02 14:41:00.128858 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.128830 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" podUID="18649e26-dba6-49f4-a32e-b9cf29365168" containerName="copy" containerID="cri-o://3e90afdf00e1faa2b0a6ba62641e318a75db37e59022a52091515916e0fd3439" gracePeriod=2 Jul 02 14:41:00.131168 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.131145 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-k7pk7/must-gather-v4lqp"] Jul 02 14:41:00.131429 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:41:00.131401 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-v4lqp\" is forbidden: User \"system:node:ip-10-0-138-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-k7pk7\": no relationship found between node 'ip-10-0-138-134.ec2.internal' and this object" podUID="18649e26-dba6-49f4-a32e-b9cf29365168" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:41:00.351032 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.350942 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k7pk7_must-gather-v4lqp_18649e26-dba6-49f4-a32e-b9cf29365168/copy/0.log" Jul 02 14:41:00.351662 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.351537 2569 generic.go:356] "Generic (PLEG): container finished" podID="18649e26-dba6-49f4-a32e-b9cf29365168" containerID="3e90afdf00e1faa2b0a6ba62641e318a75db37e59022a52091515916e0fd3439" exitCode=143 Jul 02 14:41:00.502883 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.502777 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k7pk7_must-gather-v4lqp_18649e26-dba6-49f4-a32e-b9cf29365168/copy/0.log" Jul 02 14:41:00.509556 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.506515 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:41:00.509556 ip-10-0-138-134 kubenswrapper[2569]: E0702 14:41:00.508849 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-v4lqp\" is forbidden: User \"system:node:ip-10-0-138-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-k7pk7\": no relationship found between node 'ip-10-0-138-134.ec2.internal' and this object" podUID="18649e26-dba6-49f4-a32e-b9cf29365168" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:41:00.528823 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.528787 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz9hk\" (UniqueName: \"kubernetes.io/projected/18649e26-dba6-49f4-a32e-b9cf29365168-kube-api-access-vz9hk\") pod \"18649e26-dba6-49f4-a32e-b9cf29365168\" (UID: \"18649e26-dba6-49f4-a32e-b9cf29365168\") " Jul 02 14:41:00.529007 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.528858 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/18649e26-dba6-49f4-a32e-b9cf29365168-must-gather-output\") pod \"18649e26-dba6-49f4-a32e-b9cf29365168\" (UID: \"18649e26-dba6-49f4-a32e-b9cf29365168\") " Jul 02 14:41:00.530481 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.530432 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18649e26-dba6-49f4-a32e-b9cf29365168-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "18649e26-dba6-49f4-a32e-b9cf29365168" (UID: "18649e26-dba6-49f4-a32e-b9cf29365168"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:41:00.538632 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.538477 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18649e26-dba6-49f4-a32e-b9cf29365168-kube-api-access-vz9hk" (OuterVolumeSpecName: "kube-api-access-vz9hk") pod "18649e26-dba6-49f4-a32e-b9cf29365168" (UID: "18649e26-dba6-49f4-a32e-b9cf29365168"). InnerVolumeSpecName "kube-api-access-vz9hk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:41:00.600223 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.600183 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18649e26-dba6-49f4-a32e-b9cf29365168" path="/var/lib/kubelet/pods/18649e26-dba6-49f4-a32e-b9cf29365168/volumes" Jul 02 14:41:00.630498 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.630462 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz9hk\" (UniqueName: \"kubernetes.io/projected/18649e26-dba6-49f4-a32e-b9cf29365168-kube-api-access-vz9hk\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:41:00.630498 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:00.630499 2569 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/18649e26-dba6-49f4-a32e-b9cf29365168-must-gather-output\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\"" Jul 02 14:41:01.359864 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:01.359827 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k7pk7_must-gather-v4lqp_18649e26-dba6-49f4-a32e-b9cf29365168/copy/0.log" Jul 02 14:41:01.360327 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:01.360276 2569 scope.go:117] "RemoveContainer" containerID="3e90afdf00e1faa2b0a6ba62641e318a75db37e59022a52091515916e0fd3439" Jul 02 14:41:01.360513 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:01.360492 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k7pk7/must-gather-v4lqp" Jul 02 14:41:01.376252 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:01.376228 2569 scope.go:117] "RemoveContainer" containerID="d704166ce68cd4ed20e07addd87776f1b2b572c67e2b928e22a30262bac35d51" Jul 02 14:41:02.873505 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:02.873405 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-q8f6r_8e9dccc2-45ba-4786-9319-e5bd22be8aaf/kube-state-metrics/0.log" Jul 02 14:41:02.898090 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:02.897997 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-q8f6r_8e9dccc2-45ba-4786-9319-e5bd22be8aaf/kube-rbac-proxy-main/0.log" Jul 02 14:41:02.921076 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:02.920967 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-q8f6r_8e9dccc2-45ba-4786-9319-e5bd22be8aaf/kube-rbac-proxy-self/0.log" Jul 02 14:41:03.008428 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.008399 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5fqzk_dc9d7792-4953-49cf-9172-f1e7fde3b839/node-exporter/0.log" Jul 02 14:41:03.028855 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.028827 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5fqzk_dc9d7792-4953-49cf-9172-f1e7fde3b839/kube-rbac-proxy/0.log" Jul 02 14:41:03.051025 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.050999 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5fqzk_dc9d7792-4953-49cf-9172-f1e7fde3b839/init-textfile/0.log" Jul 02 14:41:03.318417 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.318389 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b8d0e20-0504-4ccd-ae6b-8801d93ea920/prometheus/0.log" Jul 02 14:41:03.342525 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.342493 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b8d0e20-0504-4ccd-ae6b-8801d93ea920/config-reloader/0.log" Jul 02 14:41:03.365349 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.365319 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b8d0e20-0504-4ccd-ae6b-8801d93ea920/thanos-sidecar/0.log" Jul 02 14:41:03.389583 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.389559 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b8d0e20-0504-4ccd-ae6b-8801d93ea920/kube-rbac-proxy-web/0.log" Jul 02 14:41:03.416812 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.416781 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b8d0e20-0504-4ccd-ae6b-8801d93ea920/kube-rbac-proxy/0.log" Jul 02 14:41:03.438977 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.438953 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b8d0e20-0504-4ccd-ae6b-8801d93ea920/kube-rbac-proxy-thanos/0.log" Jul 02 14:41:03.460991 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.460954 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b8d0e20-0504-4ccd-ae6b-8801d93ea920/init-config-reloader/0.log" Jul 02 14:41:03.491822 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.491787 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-94sdj_1ffc614e-9b60-45ce-aeef-9cf06cadc6ad/prometheus-operator/0.log" Jul 02 14:41:03.513713 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.513676 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-94sdj_1ffc614e-9b60-45ce-aeef-9cf06cadc6ad/kube-rbac-proxy/0.log" Jul 02 14:41:03.543135 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:03.543077 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-tdkxj_96a562da-b1d3-4ec9-b253-3d7e69b4e609/prometheus-operator-admission-webhook/0.log" Jul 02 14:41:06.493509 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.493470 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g"] Jul 02 14:41:06.494047 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.493791 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18649e26-dba6-49f4-a32e-b9cf29365168" containerName="gather" Jul 02 14:41:06.494047 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.493810 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="18649e26-dba6-49f4-a32e-b9cf29365168" containerName="gather" Jul 02 14:41:06.494047 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.493835 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18649e26-dba6-49f4-a32e-b9cf29365168" containerName="copy" Jul 02 14:41:06.494047 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.493843 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="18649e26-dba6-49f4-a32e-b9cf29365168" containerName="copy" Jul 02 14:41:06.494047 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.493917 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="18649e26-dba6-49f4-a32e-b9cf29365168" containerName="copy" Jul 02 14:41:06.494047 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.493929 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="18649e26-dba6-49f4-a32e-b9cf29365168" containerName="gather" Jul 02 14:41:06.499406 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.499379 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.505299 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.505274 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g"] Jul 02 14:41:06.591864 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.591828 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-podres\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.592075 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.592049 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-lib-modules\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.592154 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.592113 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-sys\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.592154 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.592148 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw9cc\" (UniqueName: \"kubernetes.io/projected/4c889fae-4fa3-4763-9057-e3d92168a548-kube-api-access-xw9cc\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.592256 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.592193 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-proc\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.693584 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.693544 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-sys\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.693770 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.693606 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9cc\" (UniqueName: \"kubernetes.io/projected/4c889fae-4fa3-4763-9057-e3d92168a548-kube-api-access-xw9cc\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.693770 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.693624 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-sys\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.693770 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.693679 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-proc\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.693770 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.693771 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-podres\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.693948 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.693795 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-lib-modules\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.693948 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.693803 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-proc\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.693948 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.693936 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-lib-modules\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.694038 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.693990 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4c889fae-4fa3-4763-9057-e3d92168a548-podres\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.701551 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.701522 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw9cc\" (UniqueName: \"kubernetes.io/projected/4c889fae-4fa3-4763-9057-e3d92168a548-kube-api-access-xw9cc\") pod \"perf-node-gather-daemonset-c777g\" (UID: \"4c889fae-4fa3-4763-9057-e3d92168a548\") " pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.813013 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.812890 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:06.967584 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:06.967552 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g"] Jul 02 14:41:06.970796 ip-10-0-138-134 kubenswrapper[2569]: W0702 14:41:06.970761 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4c889fae_4fa3_4763_9057_e3d92168a548.slice/crio-369c73e0fd6ae9c545469ea49d108fa887114069b7c0e85577ff510ca7c581ef WatchSource:0}: Error finding container 369c73e0fd6ae9c545469ea49d108fa887114069b7c0e85577ff510ca7c581ef: Status 404 returned error can't find the container with id 369c73e0fd6ae9c545469ea49d108fa887114069b7c0e85577ff510ca7c581ef Jul 02 14:41:07.397808 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:07.397772 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" event={"ID":"4c889fae-4fa3-4763-9057-e3d92168a548","Type":"ContainerStarted","Data":"baf7ff50d127504771f88fe40f212015e0b81c92e1d22469e89c2462eeb61b44"} Jul 02 14:41:07.397982 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:07.397860 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" event={"ID":"4c889fae-4fa3-4763-9057-e3d92168a548","Type":"ContainerStarted","Data":"369c73e0fd6ae9c545469ea49d108fa887114069b7c0e85577ff510ca7c581ef"} Jul 02 14:41:07.397982 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:07.397978 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:07.415340 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:07.415292 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" podStartSLOduration=1.415278205 podStartE2EDuration="1.415278205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:41:07.414586521 +0000 UTC m=+1531.289669980" watchObservedRunningTime="2026-07-02 14:41:07.415278205 +0000 UTC m=+1531.290361659" Jul 02 14:41:07.614279 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:07.614251 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2bkg2_5ba9c566-6099-40fd-9867-2e2dea90f315/dns/0.log" Jul 02 14:41:07.636685 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:07.636656 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2bkg2_5ba9c566-6099-40fd-9867-2e2dea90f315/kube-rbac-proxy/0.log" Jul 02 14:41:07.811775 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:07.811708 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mf4f7_ea0e1336-22b5-4c5c-b34d-e4a469f0bc60/dns-node-resolver/0.log" Jul 02 14:41:08.272726 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:08.272694 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-67b7477fd9-w82tg_c763bfc2-3408-4bd8-adc4-6d1ee07d90f1/registry/0.log" Jul 02 14:41:08.292137 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:08.292110 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8jkcr_8cc01e91-17e6-465e-b77d-acceac9e4a27/node-ca/0.log" Jul 02 14:41:09.543022 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:09.542989 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pkk26_3ee9e4a2-e775-4a5e-b02b-021c2c31a1dc/serve-healthcheck-canary/0.log" Jul 02 14:41:10.054349 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:10.054322 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cbz48_cf882eea-6e9f-40be-8a75-249601c205de/kube-rbac-proxy/0.log" Jul 02 14:41:10.075410 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:10.075385 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cbz48_cf882eea-6e9f-40be-8a75-249601c205de/exporter/0.log" Jul 02 14:41:10.095977 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:10.095953 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cbz48_cf882eea-6e9f-40be-8a75-249601c205de/extractor/0.log" Jul 02 14:41:12.418537 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:12.418494 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-l2xnv_79a2d70b-89ef-466e-8ffe-da4a14883ce4/manager/0.log" Jul 02 14:41:12.456039 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:12.456010 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-tjkzk_e100c45b-f8f8-4987-add2-ad05134f3b1d/s3-init/0.log" Jul 02 14:41:12.472379 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:12.472355 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-tjkzk_e100c45b-f8f8-4987-add2-ad05134f3b1d/download-hf-model/0.log" Jul 02 14:41:13.411698 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:13.411671 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-52qrt/perf-node-gather-daemonset-c777g" Jul 02 14:41:18.377320 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:18.377285 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhddh_2b739ac8-5c98-4d5a-a3bf-133c98962b46/kube-multus-additional-cni-plugins/0.log" Jul 02 14:41:18.402714 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:18.402686 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhddh_2b739ac8-5c98-4d5a-a3bf-133c98962b46/egress-router-binary-copy/0.log" Jul 02 14:41:18.424727 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:18.424705 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhddh_2b739ac8-5c98-4d5a-a3bf-133c98962b46/cni-plugins/0.log" Jul 02 14:41:18.446514 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:18.446485 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhddh_2b739ac8-5c98-4d5a-a3bf-133c98962b46/bond-cni-plugin/0.log" Jul 02 14:41:18.467966 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:18.467944 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhddh_2b739ac8-5c98-4d5a-a3bf-133c98962b46/routeoverride-cni/0.log" Jul 02 14:41:18.489872 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:18.489853 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhddh_2b739ac8-5c98-4d5a-a3bf-133c98962b46/whereabouts-cni-bincopy/0.log" Jul 02 14:41:18.511726 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:18.511703 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhddh_2b739ac8-5c98-4d5a-a3bf-133c98962b46/whereabouts-cni/0.log" Jul 02 14:41:18.542651 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:18.542626 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-crjcx_4948db6d-308b-4367-927b-47c82a9ad85d/kube-multus/0.log" Jul 02 14:41:18.679075 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:18.679004 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9z5n8_74858031-b9e7-4b68-83b7-9cd8e9b22824/network-metrics-daemon/0.log" Jul 02 14:41:18.704530 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:18.704506 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9z5n8_74858031-b9e7-4b68-83b7-9cd8e9b22824/kube-rbac-proxy/0.log" Jul 02 14:41:19.506837 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:19.506802 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-controller/0.log" Jul 02 14:41:19.525587 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:19.525564 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/0.log" Jul 02 14:41:19.525758 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:19.525747 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovn-acl-logging/1.log" Jul 02 14:41:19.546369 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:19.546339 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/kube-rbac-proxy-node/0.log" Jul 02 14:41:19.568722 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:19.568697 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/kube-rbac-proxy-ovn-metrics/0.log" Jul 02 14:41:19.593938 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:19.593909 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/northd/0.log" Jul 02 14:41:19.615830 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:19.615810 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/nbdb/0.log" Jul 02 14:41:19.640560 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:19.640539 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/sbdb/0.log" Jul 02 14:41:19.685637 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:19.685612 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74t6q_fb98cb35-0b85-40e4-9a88-54f73548fb3d/ovnkube-controller/0.log" Jul 02 14:41:21.198091 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:21.198064 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-rzbpn_b5a88716-1c0d-47f1-95b5-50c0b04d89c1/network-check-target-container/0.log" Jul 02 14:41:22.167876 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:22.167851 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-kr8t2_1942e8f3-282c-427b-af88-9bf72887e1e6/iptables-alerter/0.log" Jul 02 14:41:22.903360 ip-10-0-138-134 kubenswrapper[2569]: I0702 14:41:22.903328 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-thntg_d625f978-90bd-4f61-b0a2-644876bcf84d/tuned/0.log"