Jul 09 17:39:43.803560 ip-10-0-133-63 systemd[1]: Starting Kubernetes Kubelet... Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.186757 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193217 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193228 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193232 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193235 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193238 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193241 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193243 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:39:44.261451 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193247 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193250 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193253 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193255 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193258 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193260 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193280 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193284 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193286 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193289 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193292 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193295 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193297 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193301 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193304 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193306 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193310 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193315 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193318 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:39:44.262237 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193321 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193324 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193326 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193329 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193332 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193335 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193337 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193340 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193342 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193345 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193347 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193352 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193356 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193359 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193363 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193365 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193368 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193370 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193373 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:39:44.262800 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193376 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193379 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193382 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193384 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193387 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193390 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193393 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193396 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193399 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193402 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193404 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193407 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193409 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193412 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193415 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193417 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193420 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193422 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193425 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193427 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:39:44.263471 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193431 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193434 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193436 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193439 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193442 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193444 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193447 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193449 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193452 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193455 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193457 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193460 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193463 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193466 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193470 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193473 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193477 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193480 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193483 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:39:44.264026 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193487 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193490 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193493 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193496 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193499 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193502 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193504 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193507 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193510 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193513 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193516 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193518 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193521 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193523 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.193526 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195890 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195902 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195906 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195909 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195912 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:39:44.264548 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195916 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195919 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195921 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195924 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195927 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195931 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195936 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195939 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195942 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195945 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195949 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195953 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195956 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195959 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195962 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195966 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195969 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195977 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195980 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:39:44.265134 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195982 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195985 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195988 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195991 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195995 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.195998 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196000 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196003 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196006 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196010 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196012 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196015 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196017 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196020 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196022 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196025 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196028 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196030 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196033 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196035 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:39:44.265774 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196038 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:39:44.266309 ip-10-0-133-63 systemd[1]: Started Kubernetes Kubelet. Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196040 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196043 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196045 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196048 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196050 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196053 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196055 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196058 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196061 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196064 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196068 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196071 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196075 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196078 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196080 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196084 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196086 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196089 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:39:44.266621 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196092 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196094 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196097 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196099 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196102 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196105 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196107 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196109 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196112 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196115 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196117 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196119 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196122 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196124 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196127 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196129 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196132 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196134 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196137 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196139 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:39:44.267557 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196143 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196145 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196148 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196150 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196153 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196155 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196158 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196160 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196163 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196166 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196170 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196172 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196175 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196177 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196180 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.196182 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197320 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197327 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197331 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197334 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:39:44.268203 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197337 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197340 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197343 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197346 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197349 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197352 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197354 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197357 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197360 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197362 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197365 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197367 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197371 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197374 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197376 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197379 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197382 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197385 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197387 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197390 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:39:44.268850 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197392 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197395 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197398 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197400 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197403 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197405 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197408 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197410 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197413 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197416 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197419 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197421 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197424 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197426 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197429 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197432 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197434 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197437 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197439 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:39:44.269509 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197442 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197445 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197447 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197450 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197453 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197456 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197459 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197462 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197464 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197469 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197472 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197475 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197478 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197481 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197483 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197486 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197489 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197491 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197496 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:39:44.270177 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197499 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197502 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197505 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197508 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197510 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197514 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197517 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197520 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197523 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197525 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197528 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197530 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197533 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197535 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197538 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197541 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197544 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197546 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197549 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197552 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:39:44.270788 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197556 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197559 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197561 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197564 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197566 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197569 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197571 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197574 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197577 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197580 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197582 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197584 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197587 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197589 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197592 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197595 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.197597 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197664 2571 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197671 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197676 2571 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197681 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 17:39:44.271397 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197685 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197688 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197693 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197699 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197703 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197706 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197710 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197713 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197716 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197719 2571 flags.go:64] FLAG: --cgroup-root="" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197722 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197725 2571 flags.go:64] FLAG: --client-ca-file="" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197728 2571 flags.go:64] FLAG: --cloud-provider="external" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197731 2571 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197736 2571 flags.go:64] FLAG: --cluster-domain="" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197739 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197742 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197745 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197748 2571 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197752 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197755 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197758 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197761 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197765 2571 flags.go:64] FLAG: --contention-profiling="false" Jul 09 17:39:44.272012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197768 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197772 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197776 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197779 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197782 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197786 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197788 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197791 2571 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197794 2571 flags.go:64] FLAG: --enable-server="true" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197797 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197801 2571 flags.go:64] FLAG: --event-burst="100" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197806 2571 flags.go:64] FLAG: --event-qps="50" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197809 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197812 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197815 2571 flags.go:64] FLAG: --eviction-hard="" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197818 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197821 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197824 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197827 2571 flags.go:64] FLAG: --eviction-soft="" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197830 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197833 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197837 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197840 2571 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197843 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197846 2571 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 17:39:44.273000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197849 2571 flags.go:64] FLAG: --feature-gates="" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197852 2571 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197855 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197858 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197861 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197864 2571 flags.go:64] FLAG: --healthz-port="10248" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197867 2571 flags.go:64] FLAG: --help="false" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197870 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-133-63.ec2.internal" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197875 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197877 2571 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197880 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197883 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197887 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197890 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197892 2571 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197895 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197905 2571 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197909 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197914 2571 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197917 2571 flags.go:64] FLAG: --kube-reserved="" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197920 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197922 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197926 2571 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197928 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 17:39:44.274149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197931 2571 flags.go:64] FLAG: --lock-file="" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197934 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197937 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197940 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197945 2571 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197949 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197952 2571 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197955 2571 flags.go:64] FLAG: --logging-format="text" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197958 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197961 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197964 2571 flags.go:64] FLAG: --manifest-url="" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197966 2571 flags.go:64] FLAG: --manifest-url-header="" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197970 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197973 2571 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197977 2571 flags.go:64] FLAG: --max-pods="110" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197980 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197984 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197987 2571 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197990 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197993 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197996 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.197999 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198005 2571 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198008 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198011 2571 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 17:39:44.547533 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198014 2571 flags.go:64] FLAG: --pod-cidr="" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198019 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198024 2571 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198027 2571 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198030 2571 flags.go:64] FLAG: --pods-per-core="0" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198033 2571 flags.go:64] FLAG: --port="10250" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198036 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198039 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e3dc528dd22759a4" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198042 2571 flags.go:64] FLAG: --qos-reserved="" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198045 2571 flags.go:64] FLAG: --read-only-port="10255" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198048 2571 flags.go:64] FLAG: --register-node="true" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198051 2571 flags.go:64] FLAG: --register-with-taints="" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198054 2571 flags.go:64] FLAG: --registry-burst="10" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198057 2571 flags.go:64] FLAG: --registry-qps="5" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198060 2571 flags.go:64] FLAG: --reserved-cpus="" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198063 2571 flags.go:64] FLAG: --reserved-memory="" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198067 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198070 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198073 2571 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198076 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198078 2571 flags.go:64] FLAG: --runonce="false" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198081 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198084 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198092 2571 flags.go:64] FLAG: --seccomp-default="false" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198094 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 17:39:44.549045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198097 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198100 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198103 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198106 2571 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198109 2571 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198112 2571 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198115 2571 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198118 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198121 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198125 2571 flags.go:64] FLAG: --system-cgroups="" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198128 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198133 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198136 2571 flags.go:64] FLAG: --tls-cert-file="" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198139 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198143 2571 flags.go:64] FLAG: --tls-min-version="" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198146 2571 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198149 2571 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198152 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198155 2571 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198158 2571 flags.go:64] FLAG: --v="2" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198162 2571 flags.go:64] FLAG: --version="false" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198166 2571 flags.go:64] FLAG: --vmodule="" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198169 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.198173 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198295 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:39:44.550050 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198300 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198303 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198306 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198309 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198311 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198316 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198319 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198321 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198325 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198329 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198332 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198335 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198338 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198341 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198344 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198347 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198350 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198353 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198355 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:39:44.550930 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198359 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198363 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198366 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198369 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198372 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198374 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198377 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198379 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198382 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198385 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198388 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198390 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198393 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198396 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198398 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198401 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198403 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198406 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198410 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198413 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:39:44.551661 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198415 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198418 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198421 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198423 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198425 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198428 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198430 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198433 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198435 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198440 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198443 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198446 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198448 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198451 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198453 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198456 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198458 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198461 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198463 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198466 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:39:44.552293 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198468 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198471 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198473 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198477 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198481 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198484 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198487 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198490 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198493 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198495 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198499 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198501 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198504 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198507 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198509 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198512 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198514 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198517 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198519 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198522 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:39:44.553375 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198524 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198529 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198531 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198534 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198537 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198539 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198542 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198544 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198546 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198549 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198552 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198554 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198557 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198560 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198563 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198566 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198569 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198571 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:39:44.554022 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.198574 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.199177 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.211684 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.211699 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.211720 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.211726 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211781 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211785 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211789 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211792 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211795 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211798 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211801 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211803 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211806 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:39:44.554651 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211809 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211812 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211814 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211817 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211820 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211823 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211826 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211829 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211832 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211834 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211837 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211840 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211842 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211845 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211847 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211850 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211855 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211858 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211861 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:39:44.555108 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211864 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211868 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211870 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211873 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211876 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211878 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211881 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211883 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211886 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211888 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211891 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211893 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211896 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211898 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211902 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211904 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211907 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211910 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211913 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211915 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:39:44.867216 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211920 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211923 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211925 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211928 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211931 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211933 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211936 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211939 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211941 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211944 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211947 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211950 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211952 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211955 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211959 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211961 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211964 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211966 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211969 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211971 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:39:44.869981 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211974 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211976 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211979 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211982 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211984 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211988 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211992 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211995 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.211997 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212000 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212003 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212005 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212009 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212011 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212014 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212016 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212019 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212022 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212025 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212029 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:39:44.870727 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212031 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212034 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212036 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212039 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212042 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212045 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212049 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212053 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212055 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212058 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212061 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.212066 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212193 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212197 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:39:44.871338 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212200 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212203 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212206 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212209 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212211 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212214 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212217 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212219 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212222 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212224 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212229 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212233 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212237 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212240 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212243 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212246 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212249 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212251 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212256 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:39:44.871908 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212258 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212261 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212264 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212285 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212288 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212291 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212293 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212297 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212299 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212302 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212305 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212307 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212310 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212313 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212316 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212318 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212321 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212324 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212326 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212329 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:39:44.872551 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212331 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212334 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212336 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212340 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212343 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212346 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212348 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212351 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212354 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212356 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212359 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212362 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212365 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212368 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212371 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212374 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212376 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212379 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212382 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212385 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:39:44.873143 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212388 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212391 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212393 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212396 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212399 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212401 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212404 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212406 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212409 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212412 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212414 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212417 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212419 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212422 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212424 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212428 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212431 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212434 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212436 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212439 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:39:44.873923 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212442 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212444 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212447 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212450 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212453 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212456 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212458 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212462 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212466 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212469 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212472 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212474 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212478 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212482 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212485 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212488 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212491 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:39:44.874531 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:44.212493 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.212498 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.212601 2571 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.214965 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.216261 2571 server.go:1019] "Starting client certificate rotation" Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.216370 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.216404 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.218931 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.220426 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.235100 2571 log.go:25] "Validated CRI v1 runtime API" Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.237239 2571 log.go:25] "Validated CRI v1 image API" Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.239069 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 17:39:44.875137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.242495 2571 fs.go:135] Filesystem UUIDs: map[51f28a8d-dc2c-4b2e-a429-218db26919b3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8d467cd5-9633-44a9-ac60-330afa588d15:/dev/nvme0n1p3] Jul 09 17:39:44.875559 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.242513 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 17:39:44.875559 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.247436 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.248006 2571 manager.go:216] Machine: {Timestamp:2026-07-09 17:39:44.246674351 +0000 UTC m=+0.332797813 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100686 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24a75d2a3c487fb468eb4ca3bb25de SystemUUID:ec24a75d-2a3c-487f-b468-eb4ca3bb25de BootID:1ce6ca6f-3909-4673-bd67-2c8afcfd1f2a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:62:64:4e:36:b9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:62:64:4e:36:b9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:10:18:2b:50:bb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.248103 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.248183 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.251034 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.251062 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-63.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.251785 2571 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.251798 2571 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.251805 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.252383 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.253712 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.256539 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.256659 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.256699 2571 kubelet.go:486] "Attempting to sync node with API server" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.256708 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.256718 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.256726 2571 kubelet.go:398] "Adding apiserver pod source" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.256736 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 17:39:44.875866 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.257084 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.257104 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.259169 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.260939 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.260969 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261124 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261138 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261146 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261152 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261162 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261170 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261644 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261651 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261658 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261664 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261678 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.261686 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.262390 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.262398 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.265491 2571 server.go:1268] "Started kubelet" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.265633 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.265713 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.265744 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.266924 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.272883 2571 server.go:310] "Adding debug handlers to kubelet server" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.273050 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.273586 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-63.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.273623 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-63.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.274076 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.277187 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2x67z" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.277726 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-63.ec2.internal.18c0afd592db62bd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-63.ec2.internal,UID:ip-10-0-133-63.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-63.ec2.internal,},FirstTimestamp:2026-07-09 17:39:44.265462461 +0000 UTC m=+0.351585917,LastTimestamp:2026-07-09 17:39:44.265462461 +0000 UTC m=+0.351585917,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-63.ec2.internal,}" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.278900 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.278924 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.279037 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.279603 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.279721 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.279734 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.279832 2571 factory.go:55] Registering systemd factory Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.279845 2571 factory.go:223] Registration of the systemd container factory successfully Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.279850 2571 reconstruct.go:97] "Volume reconstruction finished" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.279861 2571 reconciler.go:29] "Reconciler: start to sync state" Jul 09 17:39:44.876428 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.280076 2571 factory.go:153] Registering CRI-O factory Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.280090 2571 factory.go:223] Registration of the crio container factory successfully Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.280136 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.280157 2571 factory.go:103] Registering Raw factory Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.280173 2571 manager.go:1196] Started watching for new ooms in manager Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.280628 2571 manager.go:349] Starting recovery of all containers Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.286614 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.286708 2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-63.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.289613 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2x67z" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.295955 2571 manager.go:354] Recovery completed Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.300016 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.301852 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.301874 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.301886 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.302375 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.302385 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.302402 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.302386 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.303132 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.303159 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.303169 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.303552 2571 policy_none.go:49] "None policy: Start" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.303564 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.303574 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.304477 2571 policy_none.go:47] "Start" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.305055 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-63.ec2.internal.18c0afd59506ce14 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-63.ec2.internal,UID:ip-10-0-133-63.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-63.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-63.ec2.internal,},FirstTimestamp:2026-07-09 17:39:44.30186242 +0000 UTC m=+0.387985876,LastTimestamp:2026-07-09 17:39:44.30186242 +0000 UTC m=+0.387985876,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-63.ec2.internal,}" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.348786 2571 manager.go:337] "Starting Device Plugin manager" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.348954 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.348965 2571 server.go:87] "Starting device plugin registration server" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.349162 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.349170 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.349294 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.349379 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.349389 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.349874 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.349938 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.452886 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.454417 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:39:44.877598 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.454451 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.454465 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.454493 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.463080 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.463101 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-63.ec2.internal\": node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.470411 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.471746 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.471778 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.471799 2571 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.471831 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.473895 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.478241 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.572975 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal"] Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.573742 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.574754 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.574789 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.574803 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.576659 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.576881 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.576915 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.577500 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.577524 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.577534 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.578178 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.578210 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.578223 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.578460 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.578767 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.578795 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.579459 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.579483 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.579493 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.581530 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0b788ab3b6278577defa639743bb22fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal\" (UID: \"0b788ab3b6278577defa639743bb22fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.581554 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b788ab3b6278577defa639743bb22fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal\" (UID: \"0b788ab3b6278577defa639743bb22fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.581574 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a96eeef141b89fa6a1cb2d723efd4cb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-63.ec2.internal\" (UID: \"1a96eeef141b89fa6a1cb2d723efd4cb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.161897 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.608886 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-63.ec2.internal\" not found" node="ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.613148 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-63.ec2.internal\" not found" node="ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.679537 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.681856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a96eeef141b89fa6a1cb2d723efd4cb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-63.ec2.internal\" (UID: \"1a96eeef141b89fa6a1cb2d723efd4cb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.681881 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0b788ab3b6278577defa639743bb22fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal\" (UID: \"0b788ab3b6278577defa639743bb22fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.681898 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b788ab3b6278577defa639743bb22fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal\" (UID: \"0b788ab3b6278577defa639743bb22fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.681946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b788ab3b6278577defa639743bb22fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal\" (UID: \"0b788ab3b6278577defa639743bb22fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.681979 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0b788ab3b6278577defa639743bb22fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal\" (UID: \"0b788ab3b6278577defa639743bb22fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.681979 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a96eeef141b89fa6a1cb2d723efd4cb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-63.ec2.internal\" (UID: \"1a96eeef141b89fa6a1cb2d723efd4cb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.780134 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.881100 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.912008 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:44.916100 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:44.981959 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.163764 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:45.086312 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.499099 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:45.186445 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.499099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.216887 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 17:39:45.499099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.217050 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 17:39:45.499099 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:45.286551 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.499099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.292626 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 17:34:45 +0000 UTC" deadline="2028-01-19 23:39:04.171793948 +0000 UTC" Jul 09 17:39:45.499099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.292665 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13421h59m18.879134196s" Jul 09 17:39:45.499099 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:45.389491 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.499099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.415809 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:39:45.499099 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:45.490949 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-63.ec2.internal\" not found" Jul 09 17:39:45.712005 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.514833 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:39:45.712005 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.579143 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.712005 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.594816 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 17:39:45.712005 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.596654 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal" Jul 09 17:39:45.712005 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.604202 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 17:39:45.712005 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:45.628480 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a96eeef141b89fa6a1cb2d723efd4cb.slice/crio-6b33235fbca142a8133a00f30721994c32d9a20aababc67a4e2861ea648c660c WatchSource:0}: Error finding container 6b33235fbca142a8133a00f30721994c32d9a20aababc67a4e2861ea648c660c: Status 404 returned error can't find the container with id 6b33235fbca142a8133a00f30721994c32d9a20aababc67a4e2861ea648c660c Jul 09 17:39:45.712005 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:45.629048 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b788ab3b6278577defa639743bb22fc.slice/crio-d147ac4b647c0c0c6a23050ff5f455266287da69648683c4a88b5e0a59ff57e8 WatchSource:0}: Error finding container d147ac4b647c0c0c6a23050ff5f455266287da69648683c4a88b5e0a59ff57e8: Status 404 returned error can't find the container with id d147ac4b647c0c0c6a23050ff5f455266287da69648683c4a88b5e0a59ff57e8 Jul 09 17:39:45.712005 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.633235 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:39:45.942690 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:45.791261 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:39:46.262180 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.259846 2571 apiserver.go:52] "Watching apiserver" Jul 09 17:39:46.280565 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.280334 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.313423 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.315163 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-2p7ff","kube-system/global-pull-secret-syncer-pdswh","openshift-cluster-node-tuning-operator/tuned-mm7d7","openshift-dns/node-resolver-nzt4v","openshift-multus/multus-fwnfj","openshift-network-operator/iptables-alerter-fvp4m","openshift-ovn-kubernetes/ovnkube-node-b547z","kube-system/konnectivity-agent-rcl6f","kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6","openshift-image-registry/node-ca-czxd6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal","openshift-multus/multus-additional-cni-plugins-px9wm","openshift-multus/network-metrics-daemon-lwssb"] Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.317049 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.319464 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.320211 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.320343 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v5wk6\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.320425 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bs8hl" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.320686 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.320831 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.320873 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.321190 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.321406 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.321442 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.321573 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.322416 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.322634 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.322836 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.323205 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.323469 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.323830 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.324025 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2v544\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.324510 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-87g9d\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.324896 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.325172 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bs8hl" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.326077 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.327988 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.328165 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.332320 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.332746 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.333057 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.333118 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 17:39:46.379661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.333349 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.333416 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.334979 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.335385 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.335764 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.335935 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.336090 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mts7w\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.336269 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kf7hl\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.336538 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mlbtz\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.336816 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.336878 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.338466 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.341632 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.342828 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.342865 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.343157 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5r8k5\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.343261 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.344595 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.366433 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.366676 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jhzg5\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.366911 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.367710 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.369641 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.370753 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 17:39:46.381461 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.370918 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rmvhx\"" Jul 09 17:39:46.385392 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.381821 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.394900 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6p6w\" (UniqueName: \"kubernetes.io/projected/f7e2b98f-c7e7-4796-b08e-820999127101-kube-api-access-w6p6w\") pod \"node-resolver-nzt4v\" (UID: \"f7e2b98f-c7e7-4796-b08e-820999127101\") " pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.394948 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-kubelet-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.394976 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-tuned\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.395000 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-cnibin\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.395037 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1654e8a4-0a47-4d4f-a3de-35a68c06bcea-serviceca\") pod \"node-ca-czxd6\" (UID: \"1654e8a4-0a47-4d4f-a3de-35a68c06bcea\") " pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.395090 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b0c760e-92c9-4e00-bf4e-bf578db74755-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.395128 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-sysconfig\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.395155 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-socket-dir-parent\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.395184 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.395216 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-var-lib-kubelet\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.395240 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-var-lib-cni-bin\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.396717 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b67de07-eb02-48c3-9e6e-31f3b83d24b7-host-slash\") pod \"iptables-alerter-fvp4m\" (UID: \"2b67de07-eb02-48c3-9e6e-31f3b83d24b7\") " pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.396798 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcgzt\" (UniqueName: \"kubernetes.io/projected/02523ead-082e-41e1-ab84-648a74b97bd2-kube-api-access-qcgzt\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.396826 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1654e8a4-0a47-4d4f-a3de-35a68c06bcea-host\") pod \"node-ca-czxd6\" (UID: \"1654e8a4-0a47-4d4f-a3de-35a68c06bcea\") " pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.396905 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-run-netns\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.482630 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.396958 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-kubernetes\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.396982 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-sysctl-conf\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397053 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-run-netns\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397079 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-var-lib-openvswitch\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397146 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-kubelet\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397172 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-etc-openvswitch\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397251 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-node-log\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397340 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-cni-bin\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397365 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-sysctl-d\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397425 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-lib-modules\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397453 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f515f8f-2a9d-40fc-8049-41438737d2bb-cni-binary-copy\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397519 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-run-multus-certs\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397546 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5890960e-4225-4cae-83d2-a5749bb9d755-dbus\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.397613 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-cni-netd\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.398097 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-system-cni-dir\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.398187 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-os-release\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.398213 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-var-lib-kubelet\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.486141 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.398313 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-run\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.422806 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-sys\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424367 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-conf-dir\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424398 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-slash\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424432 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-run-systemd\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424457 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-ovnkube-script-lib\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424481 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmzh7\" (UniqueName: \"kubernetes.io/projected/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-kube-api-access-vmzh7\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424518 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r67pr\" (UniqueName: \"kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr\") pod \"network-check-target-2p7ff\" (UID: \"c86cda2a-444c-431e-b014-9cef96fe3d8e\") " pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424543 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5890960e-4225-4cae-83d2-a5749bb9d755-kubelet-config\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424567 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51-konnectivity-ca\") pod \"konnectivity-agent-rcl6f\" (UID: \"d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51\") " pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424588 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-run-openvswitch\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424610 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7e2b98f-c7e7-4796-b08e-820999127101-hosts-file\") pod \"node-resolver-nzt4v\" (UID: \"f7e2b98f-c7e7-4796-b08e-820999127101\") " pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424635 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-run-ovn\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424669 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b0c760e-92c9-4e00-bf4e-bf578db74755-cni-binary-copy\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424697 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b0c760e-92c9-4e00-bf4e-bf578db74755-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424722 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-sys-fs\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.618175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424746 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3fd70bf-947f-4743-a769-f926c81ebf21-tmp\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424770 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-hostroot\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424796 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-daemon-config\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424853 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-etc-kubernetes\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424890 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424915 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-registration-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424942 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-host\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424966 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-log-socket\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.424990 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-cnibin\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425015 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-tuning-conf-dir\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425037 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-modprobe-d\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425060 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-env-overrides\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425085 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-systemd\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425429 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-os-release\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425457 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-var-lib-cni-multus\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425544 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t78gx\" (UniqueName: \"kubernetes.io/projected/2b67de07-eb02-48c3-9e6e-31f3b83d24b7-kube-api-access-t78gx\") pod \"iptables-alerter-fvp4m\" (UID: \"2b67de07-eb02-48c3-9e6e-31f3b83d24b7\") " pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.619387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425571 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v66dl\" (UniqueName: \"kubernetes.io/projected/1654e8a4-0a47-4d4f-a3de-35a68c06bcea-kube-api-access-v66dl\") pod \"node-ca-czxd6\" (UID: \"1654e8a4-0a47-4d4f-a3de-35a68c06bcea\") " pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425630 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-socket-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425661 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-etc-selinux\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425725 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk8tt\" (UniqueName: \"kubernetes.io/projected/2d964e80-05f9-401a-ae9f-47e828370872-kube-api-access-zk8tt\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425749 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-systemd-units\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425835 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-ovnkube-config\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425872 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-ovn-node-metrics-cert\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425900 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-device-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425950 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt4bg\" (UniqueName: \"kubernetes.io/projected/d3fd70bf-947f-4743-a769-f926c81ebf21-kube-api-access-qt4bg\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.425994 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-cni-dir\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.426025 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b67de07-eb02-48c3-9e6e-31f3b83d24b7-iptables-alerter-script\") pod \"iptables-alerter-fvp4m\" (UID: \"2b67de07-eb02-48c3-9e6e-31f3b83d24b7\") " pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.426076 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm7xj\" (UniqueName: \"kubernetes.io/projected/1b0c760e-92c9-4e00-bf4e-bf578db74755-kube-api-access-xm7xj\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.426176 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e2b98f-c7e7-4796-b08e-820999127101-tmp-dir\") pod \"node-resolver-nzt4v\" (UID: \"f7e2b98f-c7e7-4796-b08e-820999127101\") " pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.426205 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-run-ovn-kubernetes\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.426255 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-system-cni-dir\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.426310 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-run-k8s-cni-cncf-io\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.620948 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.426376 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmpxl\" (UniqueName: \"kubernetes.io/projected/0f515f8f-2a9d-40fc-8049-41438737d2bb-kube-api-access-vmpxl\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.426427 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.426452 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51-agent-certs\") pod \"konnectivity-agent-rcl6f\" (UID: \"d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51\") " pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.486956 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" event={"ID":"0b788ab3b6278577defa639743bb22fc","Type":"ContainerStarted","Data":"d147ac4b647c0c0c6a23050ff5f455266287da69648683c4a88b5e0a59ff57e8"} Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.488559 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal" event={"ID":"1a96eeef141b89fa6a1cb2d723efd4cb","Type":"ContainerStarted","Data":"6b33235fbca142a8133a00f30721994c32d9a20aababc67a4e2861ea648c660c"} Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527483 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm7xj\" (UniqueName: \"kubernetes.io/projected/1b0c760e-92c9-4e00-bf4e-bf578db74755-kube-api-access-xm7xj\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527524 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e2b98f-c7e7-4796-b08e-820999127101-tmp-dir\") pod \"node-resolver-nzt4v\" (UID: \"f7e2b98f-c7e7-4796-b08e-820999127101\") " pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527552 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-run-ovn-kubernetes\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527577 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-system-cni-dir\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527602 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-run-k8s-cni-cncf-io\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527626 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmpxl\" (UniqueName: \"kubernetes.io/projected/0f515f8f-2a9d-40fc-8049-41438737d2bb-kube-api-access-vmpxl\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527652 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527676 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51-agent-certs\") pod \"konnectivity-agent-rcl6f\" (UID: \"d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51\") " pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527704 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6p6w\" (UniqueName: \"kubernetes.io/projected/f7e2b98f-c7e7-4796-b08e-820999127101-kube-api-access-w6p6w\") pod \"node-resolver-nzt4v\" (UID: \"f7e2b98f-c7e7-4796-b08e-820999127101\") " pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527734 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-kubelet-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527757 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-tuned\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527782 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-cnibin\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.621839 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527808 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1654e8a4-0a47-4d4f-a3de-35a68c06bcea-serviceca\") pod \"node-ca-czxd6\" (UID: \"1654e8a4-0a47-4d4f-a3de-35a68c06bcea\") " pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527835 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b0c760e-92c9-4e00-bf4e-bf578db74755-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527863 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-sysconfig\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527889 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-socket-dir-parent\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527915 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527946 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-var-lib-kubelet\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.527977 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-var-lib-cni-bin\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528001 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b67de07-eb02-48c3-9e6e-31f3b83d24b7-host-slash\") pod \"iptables-alerter-fvp4m\" (UID: \"2b67de07-eb02-48c3-9e6e-31f3b83d24b7\") " pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528027 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcgzt\" (UniqueName: \"kubernetes.io/projected/02523ead-082e-41e1-ab84-648a74b97bd2-kube-api-access-qcgzt\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528051 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1654e8a4-0a47-4d4f-a3de-35a68c06bcea-host\") pod \"node-ca-czxd6\" (UID: \"1654e8a4-0a47-4d4f-a3de-35a68c06bcea\") " pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528076 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-run-netns\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528108 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-kubernetes\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528141 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-sysctl-conf\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528178 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-run-netns\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528203 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-var-lib-openvswitch\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528227 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-kubelet\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528243 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-var-lib-cni-bin\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.623019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528249 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-cnibin\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528251 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-etc-openvswitch\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528326 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b67de07-eb02-48c3-9e6e-31f3b83d24b7-host-slash\") pod \"iptables-alerter-fvp4m\" (UID: \"2b67de07-eb02-48c3-9e6e-31f3b83d24b7\") " pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528291 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-etc-openvswitch\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528388 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-node-log\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528837 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1654e8a4-0a47-4d4f-a3de-35a68c06bcea-serviceca\") pod \"node-ca-czxd6\" (UID: \"1654e8a4-0a47-4d4f-a3de-35a68c06bcea\") " pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528876 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1654e8a4-0a47-4d4f-a3de-35a68c06bcea-host\") pod \"node-ca-czxd6\" (UID: \"1654e8a4-0a47-4d4f-a3de-35a68c06bcea\") " pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528942 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-run-netns\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528963 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b0c760e-92c9-4e00-bf4e-bf578db74755-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528997 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-kubernetes\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529029 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-sysconfig\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529129 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e2b98f-c7e7-4796-b08e-820999127101-tmp-dir\") pod \"node-resolver-nzt4v\" (UID: \"f7e2b98f-c7e7-4796-b08e-820999127101\") " pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529127 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-sysctl-conf\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529157 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-socket-dir-parent\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529187 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-run-netns\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529201 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529211 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-system-cni-dir\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529237 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-kubelet\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.624023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529254 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-var-lib-kubelet\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529274 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-var-lib-openvswitch\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.528326 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-node-log\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529254 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-run-k8s-cni-cncf-io\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529311 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-cni-bin\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529327 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-run-ovn-kubernetes\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529338 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-sysctl-d\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529388 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-lib-modules\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529414 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f515f8f-2a9d-40fc-8049-41438737d2bb-cni-binary-copy\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529442 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-run-multus-certs\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529468 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5890960e-4225-4cae-83d2-a5749bb9d755-dbus\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529493 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-cni-netd\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529518 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-system-cni-dir\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529545 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-os-release\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529570 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-var-lib-kubelet\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529595 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-run\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529621 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-sys\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529644 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-conf-dir\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.625007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529667 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-slash\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529695 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-run-systemd\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529719 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-ovnkube-script-lib\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529746 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmzh7\" (UniqueName: \"kubernetes.io/projected/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-kube-api-access-vmzh7\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529774 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67pr\" (UniqueName: \"kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr\") pod \"network-check-target-2p7ff\" (UID: \"c86cda2a-444c-431e-b014-9cef96fe3d8e\") " pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529800 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5890960e-4225-4cae-83d2-a5749bb9d755-kubelet-config\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529826 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51-konnectivity-ca\") pod \"konnectivity-agent-rcl6f\" (UID: \"d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51\") " pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529834 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-os-release\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529546 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-kubelet-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529855 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-run-openvswitch\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529874 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-slash\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529884 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7e2b98f-c7e7-4796-b08e-820999127101-hosts-file\") pod \"node-resolver-nzt4v\" (UID: \"f7e2b98f-c7e7-4796-b08e-820999127101\") " pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529892 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-var-lib-kubelet\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529920 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-cni-bin\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529930 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-run-ovn\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529959 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b0c760e-92c9-4e00-bf4e-bf578db74755-cni-binary-copy\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529962 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7e2b98f-c7e7-4796-b08e-820999127101-hosts-file\") pod \"node-resolver-nzt4v\" (UID: \"f7e2b98f-c7e7-4796-b08e-820999127101\") " pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.626231 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529988 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b0c760e-92c9-4e00-bf4e-bf578db74755-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530010 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-run-systemd\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530016 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-sys-fs\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530036 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-sysctl-d\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530046 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3fd70bf-947f-4743-a769-f926c81ebf21-tmp\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530061 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-sys-fs\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530074 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-hostroot\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530102 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-daemon-config\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530118 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-run\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530126 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-lib-modules\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530134 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-etc-kubernetes\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530171 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-conf-dir\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530175 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-sys\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530183 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-etc-kubernetes\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530223 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-run-multus-certs\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530230 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5890960e-4225-4cae-83d2-a5749bb9d755-dbus\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530266 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-run-ovn\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530665 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5890960e-4225-4cae-83d2-a5749bb9d755-kubelet-config\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:46.627099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530704 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b0c760e-92c9-4e00-bf4e-bf578db74755-cni-binary-copy\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530848 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-ovnkube-script-lib\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.530912 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-host-cni-netd\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.530993 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.531055 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret podName:5890960e-4225-4cae-83d2-a5749bb9d755 nodeName:}" failed. No retries permitted until 2026-07-09 17:39:47.031036086 +0000 UTC m=+3.117159554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret") pod "global-pull-secret-syncer-pdswh" (UID: "5890960e-4225-4cae-83d2-a5749bb9d755") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531173 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b0c760e-92c9-4e00-bf4e-bf578db74755-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531198 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-system-cni-dir\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.529813 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531227 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51-konnectivity-ca\") pod \"konnectivity-agent-rcl6f\" (UID: \"d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51\") " pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531250 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-hostroot\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531280 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531312 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-registration-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531339 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-host\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531390 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-log-socket\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531417 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-cnibin\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531444 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-tuning-conf-dir\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531483 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-modprobe-d\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.628015 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531508 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-env-overrides\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531555 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-systemd\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531581 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-os-release\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531617 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-var-lib-cni-multus\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531645 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t78gx\" (UniqueName: \"kubernetes.io/projected/2b67de07-eb02-48c3-9e6e-31f3b83d24b7-kube-api-access-t78gx\") pod \"iptables-alerter-fvp4m\" (UID: \"2b67de07-eb02-48c3-9e6e-31f3b83d24b7\") " pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531668 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v66dl\" (UniqueName: \"kubernetes.io/projected/1654e8a4-0a47-4d4f-a3de-35a68c06bcea-kube-api-access-v66dl\") pod \"node-ca-czxd6\" (UID: \"1654e8a4-0a47-4d4f-a3de-35a68c06bcea\") " pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531693 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-socket-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531717 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-etc-selinux\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531756 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk8tt\" (UniqueName: \"kubernetes.io/projected/2d964e80-05f9-401a-ae9f-47e828370872-kube-api-access-zk8tt\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531799 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-systemd-units\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531822 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-host-var-lib-cni-multus\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531833 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-modprobe-d\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531839 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-ovnkube-config\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531886 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-ovn-node-metrics-cert\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531911 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-device-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531976 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-device-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.531981 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-run-openvswitch\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.628764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532030 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt4bg\" (UniqueName: \"kubernetes.io/projected/d3fd70bf-947f-4743-a769-f926c81ebf21-kube-api-access-qt4bg\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532064 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-socket-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532067 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-cni-dir\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532117 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b67de07-eb02-48c3-9e6e-31f3b83d24b7-iptables-alerter-script\") pod \"iptables-alerter-fvp4m\" (UID: \"2b67de07-eb02-48c3-9e6e-31f3b83d24b7\") " pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532129 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-cni-dir\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532312 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-ovnkube-config\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532453 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f515f8f-2a9d-40fc-8049-41438737d2bb-cni-binary-copy\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532440 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-etc-selinux\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.532578 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532658 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f515f8f-2a9d-40fc-8049-41438737d2bb-multus-daemon-config\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532700 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-env-overrides\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532732 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-systemd\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.532745 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs podName:02523ead-082e-41e1-ab84-648a74b97bd2 nodeName:}" failed. No retries permitted until 2026-07-09 17:39:47.032724091 +0000 UTC m=+3.118847540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs") pod "network-metrics-daemon-lwssb" (UID: "02523ead-082e-41e1-ab84-648a74b97bd2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532726 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-systemd-units\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532799 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-log-socket\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532816 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f515f8f-2a9d-40fc-8049-41438737d2bb-os-release\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532835 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2d964e80-05f9-401a-ae9f-47e828370872-registration-dir\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.629643 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532847 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3fd70bf-947f-4743-a769-f926c81ebf21-host\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532886 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-cnibin\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.532901 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b0c760e-92c9-4e00-bf4e-bf578db74755-tuning-conf-dir\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.533463 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b67de07-eb02-48c3-9e6e-31f3b83d24b7-iptables-alerter-script\") pod \"iptables-alerter-fvp4m\" (UID: \"2b67de07-eb02-48c3-9e6e-31f3b83d24b7\") " pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.535813 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3fd70bf-947f-4743-a769-f926c81ebf21-tmp\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.535846 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3fd70bf-947f-4743-a769-f926c81ebf21-etc-tuned\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.536118 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51-agent-certs\") pod \"konnectivity-agent-rcl6f\" (UID: \"d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51\") " pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.536852 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-ovn-node-metrics-cert\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.561667 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm7xj\" (UniqueName: \"kubernetes.io/projected/1b0c760e-92c9-4e00-bf4e-bf578db74755-kube-api-access-xm7xj\") pod \"multus-additional-cni-plugins-px9wm\" (UID: \"1b0c760e-92c9-4e00-bf4e-bf578db74755\") " pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.565316 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt4bg\" (UniqueName: \"kubernetes.io/projected/d3fd70bf-947f-4743-a769-f926c81ebf21-kube-api-access-qt4bg\") pod \"tuned-mm7d7\" (UID: \"d3fd70bf-947f-4743-a769-f926c81ebf21\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.590463 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.590492 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.590506 2571 projected.go:196] Error preparing data for projected volume kube-api-access-r67pr for pod openshift-network-diagnostics/network-check-target-2p7ff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:46.590594 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr podName:c86cda2a-444c-431e-b014-9cef96fe3d8e nodeName:}" failed. No retries permitted until 2026-07-09 17:39:47.090565244 +0000 UTC m=+3.176688691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r67pr" (UniqueName: "kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr") pod "network-check-target-2p7ff" (UID: "c86cda2a-444c-431e-b014-9cef96fe3d8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.594506 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6p6w\" (UniqueName: \"kubernetes.io/projected/f7e2b98f-c7e7-4796-b08e-820999127101-kube-api-access-w6p6w\") pod \"node-resolver-nzt4v\" (UID: \"f7e2b98f-c7e7-4796-b08e-820999127101\") " pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.594570 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmzh7\" (UniqueName: \"kubernetes.io/projected/fede05a0-241f-47c0-aa3b-db1a5ccdbe63-kube-api-access-vmzh7\") pod \"ovnkube-node-b547z\" (UID: \"fede05a0-241f-47c0-aa3b-db1a5ccdbe63\") " pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.595008 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmpxl\" (UniqueName: \"kubernetes.io/projected/0f515f8f-2a9d-40fc-8049-41438737d2bb-kube-api-access-vmpxl\") pod \"multus-fwnfj\" (UID: \"0f515f8f-2a9d-40fc-8049-41438737d2bb\") " pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.630494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.595186 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v66dl\" (UniqueName: \"kubernetes.io/projected/1654e8a4-0a47-4d4f-a3de-35a68c06bcea-kube-api-access-v66dl\") pod \"node-ca-czxd6\" (UID: \"1654e8a4-0a47-4d4f-a3de-35a68c06bcea\") " pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.631163 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.595913 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk8tt\" (UniqueName: \"kubernetes.io/projected/2d964e80-05f9-401a-ae9f-47e828370872-kube-api-access-zk8tt\") pod \"aws-ebs-csi-driver-node-td7n6\" (UID: \"2d964e80-05f9-401a-ae9f-47e828370872\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.631163 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.596824 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t78gx\" (UniqueName: \"kubernetes.io/projected/2b67de07-eb02-48c3-9e6e-31f3b83d24b7-kube-api-access-t78gx\") pod \"iptables-alerter-fvp4m\" (UID: \"2b67de07-eb02-48c3-9e6e-31f3b83d24b7\") " pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.631163 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.612031 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcgzt\" (UniqueName: \"kubernetes.io/projected/02523ead-082e-41e1-ab84-648a74b97bd2-kube-api-access-qcgzt\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:46.635220 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.635182 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:39:46.646823 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:46.646795 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfede05a0_241f_47c0_aa3b_db1a5ccdbe63.slice/crio-6fc0b97b2542840cae63f1da5a8c2f44426f7380aef784d6b15839b4bc0f9262 WatchSource:0}: Error finding container 6fc0b97b2542840cae63f1da5a8c2f44426f7380aef784d6b15839b4bc0f9262: Status 404 returned error can't find the container with id 6fc0b97b2542840cae63f1da5a8c2f44426f7380aef784d6b15839b4bc0f9262 Jul 09 17:39:46.676944 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.676912 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-px9wm" Jul 09 17:39:46.691541 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.691037 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" Jul 09 17:39:46.698211 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.698185 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nzt4v" Jul 09 17:39:46.701192 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:46.701162 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3fd70bf_947f_4743_a769_f926c81ebf21.slice/crio-dedc6a1b730f60f14490b712f624903966f1664a4a01815f6b2ba279b29bec52 WatchSource:0}: Error finding container dedc6a1b730f60f14490b712f624903966f1664a4a01815f6b2ba279b29bec52: Status 404 returned error can't find the container with id dedc6a1b730f60f14490b712f624903966f1664a4a01815f6b2ba279b29bec52 Jul 09 17:39:46.709278 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:46.709251 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e2b98f_c7e7_4796_b08e_820999127101.slice/crio-86cafe4183219cb19b66365f1a53edf42c9893e95c2cf6e8ada1ad75f9474094 WatchSource:0}: Error finding container 86cafe4183219cb19b66365f1a53edf42c9893e95c2cf6e8ada1ad75f9474094: Status 404 returned error can't find the container with id 86cafe4183219cb19b66365f1a53edf42c9893e95c2cf6e8ada1ad75f9474094 Jul 09 17:39:46.723862 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.723682 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fwnfj" Jul 09 17:39:46.733655 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:46.733631 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f515f8f_2a9d_40fc_8049_41438737d2bb.slice/crio-101d14f339f02f9449897f336b45992929737894e36c2064fa4d5af4274ea938 WatchSource:0}: Error finding container 101d14f339f02f9449897f336b45992929737894e36c2064fa4d5af4274ea938: Status 404 returned error can't find the container with id 101d14f339f02f9449897f336b45992929737894e36c2064fa4d5af4274ea938 Jul 09 17:39:46.738112 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.737953 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fvp4m" Jul 09 17:39:46.746980 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.746959 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" Jul 09 17:39:46.747172 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:46.747144 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b67de07_eb02_48c3_9e6e_31f3b83d24b7.slice/crio-f4e843d4e7b9b9fc81fa0ed8218b9f6a7011e29687cdf7c9c63684495316b534 WatchSource:0}: Error finding container f4e843d4e7b9b9fc81fa0ed8218b9f6a7011e29687cdf7c9c63684495316b534: Status 404 returned error can't find the container with id f4e843d4e7b9b9fc81fa0ed8218b9f6a7011e29687cdf7c9c63684495316b534 Jul 09 17:39:46.755796 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.755577 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-czxd6" Jul 09 17:39:46.757086 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:46.757063 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d964e80_05f9_401a_ae9f_47e828370872.slice/crio-bedaf2d5442c3b2546bc0499ec4852faabc9f085be9be68c6178923a0949a719 WatchSource:0}: Error finding container bedaf2d5442c3b2546bc0499ec4852faabc9f085be9be68c6178923a0949a719: Status 404 returned error can't find the container with id bedaf2d5442c3b2546bc0499ec4852faabc9f085be9be68c6178923a0949a719 Jul 09 17:39:46.761462 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.761443 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:39:46.769843 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:46.769817 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1654e8a4_0a47_4d4f_a3de_35a68c06bcea.slice/crio-a620c2aea6fbcfe543c3ebab00eadabc3a8f90776585e77bfe3ae4ee9d5cd43c WatchSource:0}: Error finding container a620c2aea6fbcfe543c3ebab00eadabc3a8f90776585e77bfe3ae4ee9d5cd43c: Status 404 returned error can't find the container with id a620c2aea6fbcfe543c3ebab00eadabc3a8f90776585e77bfe3ae4ee9d5cd43c Jul 09 17:39:46.773702 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:39:46.773672 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a5d6b3_195b_4cbf_a5d6_7c7de44f3d51.slice/crio-45b074e6c4cdee9a98248c85d7e30303a4ad67a095bdf3c0161f14e17e4c60ad WatchSource:0}: Error finding container 45b074e6c4cdee9a98248c85d7e30303a4ad67a095bdf3c0161f14e17e4c60ad: Status 404 returned error can't find the container with id 45b074e6c4cdee9a98248c85d7e30303a4ad67a095bdf3c0161f14e17e4c60ad Jul 09 17:39:46.795247 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:46.795200 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:39:47.034819 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.034738 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:47.034819 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.034783 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:47.035040 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:47.034920 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:39:47.035040 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:47.034977 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret podName:5890960e-4225-4cae-83d2-a5749bb9d755 nodeName:}" failed. No retries permitted until 2026-07-09 17:39:48.03496292 +0000 UTC m=+4.121086366 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret") pod "global-pull-secret-syncer-pdswh" (UID: "5890960e-4225-4cae-83d2-a5749bb9d755") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:39:47.035040 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:47.034921 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:39:47.035188 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:47.035085 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs podName:02523ead-082e-41e1-ab84-648a74b97bd2 nodeName:}" failed. No retries permitted until 2026-07-09 17:39:48.035065533 +0000 UTC m=+4.121188993 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs") pod "network-metrics-daemon-lwssb" (UID: "02523ead-082e-41e1-ab84-648a74b97bd2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:39:47.136045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.136011 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67pr\" (UniqueName: \"kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr\") pod \"network-check-target-2p7ff\" (UID: \"c86cda2a-444c-431e-b014-9cef96fe3d8e\") " pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:47.136230 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:47.136208 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:39:47.136306 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:47.136242 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:39:47.136306 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:47.136257 2571 projected.go:196] Error preparing data for projected volume kube-api-access-r67pr for pod openshift-network-diagnostics/network-check-target-2p7ff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:39:47.136399 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:47.136345 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr podName:c86cda2a-444c-431e-b014-9cef96fe3d8e nodeName:}" failed. No retries permitted until 2026-07-09 17:39:48.13632356 +0000 UTC m=+4.222447026 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r67pr" (UniqueName: "kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr") pod "network-check-target-2p7ff" (UID: "c86cda2a-444c-431e-b014-9cef96fe3d8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:39:47.333917 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.333747 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:34:46 +0000 UTC" deadline="2028-01-21 02:32:52.853556911 +0000 UTC" Jul 09 17:39:47.333917 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.333810 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13448h53m5.519752173s" Jul 09 17:39:47.472095 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.472058 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:47.472327 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:47.472210 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:39:47.472327 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.472264 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:47.472469 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:47.472397 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:39:47.492882 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.492790 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fvp4m" event={"ID":"2b67de07-eb02-48c3-9e6e-31f3b83d24b7","Type":"ContainerStarted","Data":"f4e843d4e7b9b9fc81fa0ed8218b9f6a7011e29687cdf7c9c63684495316b534"} Jul 09 17:39:47.494528 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.494499 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px9wm" event={"ID":"1b0c760e-92c9-4e00-bf4e-bf578db74755","Type":"ContainerStarted","Data":"11e997795ebab50200a6341569c4168e77e70c9ab9673c63c43e9089123db6bf"} Jul 09 17:39:47.496591 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.496525 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-czxd6" event={"ID":"1654e8a4-0a47-4d4f-a3de-35a68c06bcea","Type":"ContainerStarted","Data":"a620c2aea6fbcfe543c3ebab00eadabc3a8f90776585e77bfe3ae4ee9d5cd43c"} Jul 09 17:39:47.500297 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.500232 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fwnfj" event={"ID":"0f515f8f-2a9d-40fc-8049-41438737d2bb","Type":"ContainerStarted","Data":"101d14f339f02f9449897f336b45992929737894e36c2064fa4d5af4274ea938"} Jul 09 17:39:47.502091 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.502062 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nzt4v" event={"ID":"f7e2b98f-c7e7-4796-b08e-820999127101","Type":"ContainerStarted","Data":"86cafe4183219cb19b66365f1a53edf42c9893e95c2cf6e8ada1ad75f9474094"} Jul 09 17:39:47.503769 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.503713 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" event={"ID":"d3fd70bf-947f-4743-a769-f926c81ebf21","Type":"ContainerStarted","Data":"dedc6a1b730f60f14490b712f624903966f1664a4a01815f6b2ba279b29bec52"} Jul 09 17:39:47.505950 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.505925 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" event={"ID":"fede05a0-241f-47c0-aa3b-db1a5ccdbe63","Type":"ContainerStarted","Data":"6fc0b97b2542840cae63f1da5a8c2f44426f7380aef784d6b15839b4bc0f9262"} Jul 09 17:39:47.508763 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.508716 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rcl6f" event={"ID":"d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51","Type":"ContainerStarted","Data":"45b074e6c4cdee9a98248c85d7e30303a4ad67a095bdf3c0161f14e17e4c60ad"} Jul 09 17:39:47.510470 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:47.510409 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" event={"ID":"2d964e80-05f9-401a-ae9f-47e828370872","Type":"ContainerStarted","Data":"bedaf2d5442c3b2546bc0499ec4852faabc9f085be9be68c6178923a0949a719"} Jul 09 17:39:48.044979 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:48.044938 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:48.045608 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:48.045001 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:48.045608 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:48.045171 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:39:48.045608 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:48.045235 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret podName:5890960e-4225-4cae-83d2-a5749bb9d755 nodeName:}" failed. No retries permitted until 2026-07-09 17:39:50.045215189 +0000 UTC m=+6.131338656 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret") pod "global-pull-secret-syncer-pdswh" (UID: "5890960e-4225-4cae-83d2-a5749bb9d755") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:39:48.045608 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:48.045460 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:39:48.045608 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:48.045518 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs podName:02523ead-082e-41e1-ab84-648a74b97bd2 nodeName:}" failed. No retries permitted until 2026-07-09 17:39:50.045502203 +0000 UTC m=+6.131625650 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs") pod "network-metrics-daemon-lwssb" (UID: "02523ead-082e-41e1-ab84-648a74b97bd2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:39:48.146159 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:48.146122 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67pr\" (UniqueName: \"kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr\") pod \"network-check-target-2p7ff\" (UID: \"c86cda2a-444c-431e-b014-9cef96fe3d8e\") " pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:48.146358 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:48.146324 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:39:48.146358 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:48.146349 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:39:48.146494 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:48.146363 2571 projected.go:196] Error preparing data for projected volume kube-api-access-r67pr for pod openshift-network-diagnostics/network-check-target-2p7ff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:39:48.146494 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:48.146421 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr podName:c86cda2a-444c-431e-b014-9cef96fe3d8e nodeName:}" failed. No retries permitted until 2026-07-09 17:39:50.146402112 +0000 UTC m=+6.232525563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r67pr" (UniqueName: "kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr") pod "network-check-target-2p7ff" (UID: "c86cda2a-444c-431e-b014-9cef96fe3d8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:39:48.335101 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:48.334970 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:34:46 +0000 UTC" deadline="2028-01-24 18:32:50.121583399 +0000 UTC" Jul 09 17:39:48.335101 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:48.335014 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13536h53m1.786577182s" Jul 09 17:39:48.475885 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:48.472951 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:48.475885 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:48.473111 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:39:49.473351 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:49.472591 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:49.473351 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:49.472727 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:39:49.473351 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:49.472825 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:49.473351 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:49.472905 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:39:49.531578 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:49.531536 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" event={"ID":"0b788ab3b6278577defa639743bb22fc","Type":"ContainerStarted","Data":"720e50a175d9dc55d019d5143845d3f9a5d5d57c104aa079e77e8a5fcc7447d2"} Jul 09 17:39:49.545974 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:49.545923 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal" event={"ID":"1a96eeef141b89fa6a1cb2d723efd4cb","Type":"ContainerStarted","Data":"1417fb05a4ee89d5a68b199a3f50308f3a7213edce4bc9d3145fb6e29855b107"} Jul 09 17:39:50.065815 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:50.065777 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:50.066020 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:50.065848 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:50.066020 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:50.065964 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:39:50.066138 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:50.066048 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs podName:02523ead-082e-41e1-ab84-648a74b97bd2 nodeName:}" failed. No retries permitted until 2026-07-09 17:39:54.066025432 +0000 UTC m=+10.152148891 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs") pod "network-metrics-daemon-lwssb" (UID: "02523ead-082e-41e1-ab84-648a74b97bd2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:39:50.066418 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:50.065967 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:39:50.066540 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:50.066479 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret podName:5890960e-4225-4cae-83d2-a5749bb9d755 nodeName:}" failed. No retries permitted until 2026-07-09 17:39:54.066456989 +0000 UTC m=+10.152580453 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret") pod "global-pull-secret-syncer-pdswh" (UID: "5890960e-4225-4cae-83d2-a5749bb9d755") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:39:50.166800 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:50.166693 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67pr\" (UniqueName: \"kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr\") pod \"network-check-target-2p7ff\" (UID: \"c86cda2a-444c-431e-b014-9cef96fe3d8e\") " pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:50.166978 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:50.166864 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:39:50.166978 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:50.166894 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:39:50.166978 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:50.166908 2571 projected.go:196] Error preparing data for projected volume kube-api-access-r67pr for pod openshift-network-diagnostics/network-check-target-2p7ff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:39:50.166978 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:50.166979 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr podName:c86cda2a-444c-431e-b014-9cef96fe3d8e nodeName:}" failed. No retries permitted until 2026-07-09 17:39:54.166953793 +0000 UTC m=+10.253077254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r67pr" (UniqueName: "kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr") pod "network-check-target-2p7ff" (UID: "c86cda2a-444c-431e-b014-9cef96fe3d8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:39:50.472590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:50.472557 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:50.472772 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:50.472710 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:39:51.473040 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:51.473002 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:51.473548 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:51.473042 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:51.473548 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:51.473170 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:39:51.473548 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:51.473348 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:39:52.476332 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:52.476244 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:52.476773 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:52.476479 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:39:52.558387 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:52.558341 2571 generic.go:356] "Generic (PLEG): container finished" podID="0b788ab3b6278577defa639743bb22fc" containerID="720e50a175d9dc55d019d5143845d3f9a5d5d57c104aa079e77e8a5fcc7447d2" exitCode=0 Jul 09 17:39:52.558558 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:52.558402 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" event={"ID":"0b788ab3b6278577defa639743bb22fc","Type":"ContainerDied","Data":"720e50a175d9dc55d019d5143845d3f9a5d5d57c104aa079e77e8a5fcc7447d2"} Jul 09 17:39:52.590137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:52.590068 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-63.ec2.internal" podStartSLOduration=6.590050563 podStartE2EDuration="6.590050563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:39:49.569651035 +0000 UTC m=+5.655774519" watchObservedRunningTime="2026-07-09 17:39:52.590050563 +0000 UTC m=+8.676174026" Jul 09 17:39:53.472882 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:53.472836 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:53.473063 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:53.472840 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:53.473063 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:53.472994 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:39:53.473188 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:53.473124 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:39:54.104648 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:54.104606 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:54.105112 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:54.104680 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:54.105112 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:54.104776 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:39:54.105112 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:54.104808 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:39:54.105112 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:54.104858 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs podName:02523ead-082e-41e1-ab84-648a74b97bd2 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:02.104837053 +0000 UTC m=+18.190960509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs") pod "network-metrics-daemon-lwssb" (UID: "02523ead-082e-41e1-ab84-648a74b97bd2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:39:54.105112 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:54.104900 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret podName:5890960e-4225-4cae-83d2-a5749bb9d755 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:02.104868304 +0000 UTC m=+18.190991753 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret") pod "global-pull-secret-syncer-pdswh" (UID: "5890960e-4225-4cae-83d2-a5749bb9d755") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:39:54.205429 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:54.205388 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67pr\" (UniqueName: \"kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr\") pod \"network-check-target-2p7ff\" (UID: \"c86cda2a-444c-431e-b014-9cef96fe3d8e\") " pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:54.205632 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:54.205615 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:39:54.205703 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:54.205642 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:39:54.205703 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:54.205654 2571 projected.go:196] Error preparing data for projected volume kube-api-access-r67pr for pod openshift-network-diagnostics/network-check-target-2p7ff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:39:54.205814 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:54.205712 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr podName:c86cda2a-444c-431e-b014-9cef96fe3d8e nodeName:}" failed. No retries permitted until 2026-07-09 17:40:02.205695226 +0000 UTC m=+18.291818693 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r67pr" (UniqueName: "kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr") pod "network-check-target-2p7ff" (UID: "c86cda2a-444c-431e-b014-9cef96fe3d8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:39:54.476547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:54.476478 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:54.476677 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:54.476613 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:39:55.472969 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:55.472940 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:55.473393 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:55.473050 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:39:55.473393 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:55.473103 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:55.473393 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:55.473184 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:39:56.472962 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:56.472478 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:56.472962 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:56.472631 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:39:57.472760 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:57.472721 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:57.472933 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:57.472875 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:39:57.472933 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:57.472913 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:57.473103 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:57.473062 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:39:58.472711 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:58.472624 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:39:58.473164 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:58.472787 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:39:59.472392 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:59.472354 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:39:59.472551 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:59.472482 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:39:59.472551 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:39:59.472530 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:39:59.472692 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:39:59.472616 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:00.472413 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:00.472379 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:00.472839 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:00.472511 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:40:01.472306 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:01.472262 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:01.472550 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:01.472411 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:40:01.472550 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:01.472471 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:01.472896 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:01.472582 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:02.168621 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:02.168578 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:02.168807 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:02.168632 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:02.168807 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:02.168740 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:40:02.168918 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:02.168817 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret podName:5890960e-4225-4cae-83d2-a5749bb9d755 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:18.168801715 +0000 UTC m=+34.254925175 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret") pod "global-pull-secret-syncer-pdswh" (UID: "5890960e-4225-4cae-83d2-a5749bb9d755") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:40:02.168918 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:02.168741 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:40:02.168918 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:02.168911 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs podName:02523ead-082e-41e1-ab84-648a74b97bd2 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:18.168890312 +0000 UTC m=+34.255013774 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs") pod "network-metrics-daemon-lwssb" (UID: "02523ead-082e-41e1-ab84-648a74b97bd2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:40:02.271736 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:02.269531 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67pr\" (UniqueName: \"kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr\") pod \"network-check-target-2p7ff\" (UID: \"c86cda2a-444c-431e-b014-9cef96fe3d8e\") " pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:02.271736 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:02.269725 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:40:02.271736 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:02.269748 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:40:02.271736 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:02.269761 2571 projected.go:196] Error preparing data for projected volume kube-api-access-r67pr for pod openshift-network-diagnostics/network-check-target-2p7ff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:40:02.271736 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:02.269828 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr podName:c86cda2a-444c-431e-b014-9cef96fe3d8e nodeName:}" failed. No retries permitted until 2026-07-09 17:40:18.269809688 +0000 UTC m=+34.355933133 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r67pr" (UniqueName: "kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr") pod "network-check-target-2p7ff" (UID: "c86cda2a-444c-431e-b014-9cef96fe3d8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:40:02.472196 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:02.472115 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:02.472371 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:02.472322 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:40:03.472517 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:03.472485 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:03.472942 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:03.472486 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:03.472942 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:03.472589 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:40:03.472942 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:03.472670 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:04.473247 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:04.473217 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:04.473676 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:04.473345 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:40:05.472408 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:05.472370 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:05.472408 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:05.472407 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:05.472624 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:05.472514 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:40:05.472670 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:05.472626 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:06.473000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.472919 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:06.473357 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:06.473058 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:40:06.586820 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.586651 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" event={"ID":"fede05a0-241f-47c0-aa3b-db1a5ccdbe63","Type":"ContainerStarted","Data":"80c6cf3eaa90e8f8f6fc679a6eb9b8f5a41864af5c8ffe7aa82e893f4ad4f1b1"} Jul 09 17:40:06.589493 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.589463 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" event={"ID":"0b788ab3b6278577defa639743bb22fc","Type":"ContainerStarted","Data":"f971ffde40dd17e0d5d7f3b7063b48929dd0e5eef040f85e4e4c145ad365e6ea"} Jul 09 17:40:06.592703 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.592292 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rcl6f" event={"ID":"d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51","Type":"ContainerStarted","Data":"44b731493e4205b90012429cacf57e049c09907ff8d4548333948ebf776180db"} Jul 09 17:40:06.594925 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.594867 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" event={"ID":"2d964e80-05f9-401a-ae9f-47e828370872","Type":"ContainerStarted","Data":"55342c966335172cd0f5a9e263f4f0fd9ac71e16a6bf8de703034b6616af83d8"} Jul 09 17:40:06.596673 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.596575 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px9wm" event={"ID":"1b0c760e-92c9-4e00-bf4e-bf578db74755","Type":"ContainerStarted","Data":"562c7ea013e2313068619d52326495c98f7ba27b1a6165d812e58f343bd127bd"} Jul 09 17:40:06.598108 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.598088 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-czxd6" event={"ID":"1654e8a4-0a47-4d4f-a3de-35a68c06bcea","Type":"ContainerStarted","Data":"1b54479c122bc2a7067a883d9b99468d94b83826b3a4e2b3804c817ceaa2f51f"} Jul 09 17:40:06.599777 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.599758 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" event={"ID":"d3fd70bf-947f-4743-a769-f926c81ebf21","Type":"ContainerStarted","Data":"e03251249e882ac3c66863fadeb98b0ec4c4b3e500d857cf85d197811127a820"} Jul 09 17:40:06.663200 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.663135 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-63.ec2.internal" podStartSLOduration=20.663110748 podStartE2EDuration="20.663110748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:06.620429502 +0000 UTC m=+22.706552968" watchObservedRunningTime="2026-07-09 17:40:06.663110748 +0000 UTC m=+22.749234215" Jul 09 17:40:06.694682 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.694626 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mm7d7" podStartSLOduration=2.079325012 podStartE2EDuration="21.69460388s" totalImagesPullingTime="19.615278868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:06.694324732 +0000 UTC m=+22.780448213" watchObservedRunningTime="2026-07-09 17:40:06.69460388 +0000 UTC m=+22.780727347" Jul 09 17:40:06.758251 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.758057 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rcl6f" podStartSLOduration=6.692283451 podStartE2EDuration="21.758037081s" totalImagesPullingTime="15.06575363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:06.757789877 +0000 UTC m=+22.843913369" watchObservedRunningTime="2026-07-09 17:40:06.758037081 +0000 UTC m=+22.844160547" Jul 09 17:40:06.758517 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:06.758481 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-czxd6" podStartSLOduration=2.230991376 podStartE2EDuration="21.758468943s" totalImagesPullingTime="19.527477567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:06.730426282 +0000 UTC m=+22.816549749" watchObservedRunningTime="2026-07-09 17:40:06.758468943 +0000 UTC m=+22.844592410" Jul 09 17:40:07.472380 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.472351 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:07.472535 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:07.472449 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:40:07.472535 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.472502 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:07.472614 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:07.472581 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:07.603281 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.603238 2571 generic.go:356] "Generic (PLEG): container finished" podID="1b0c760e-92c9-4e00-bf4e-bf578db74755" containerID="562c7ea013e2313068619d52326495c98f7ba27b1a6165d812e58f343bd127bd" exitCode=0 Jul 09 17:40:07.604073 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.603315 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px9wm" event={"ID":"1b0c760e-92c9-4e00-bf4e-bf578db74755","Type":"ContainerDied","Data":"562c7ea013e2313068619d52326495c98f7ba27b1a6165d812e58f343bd127bd"} Jul 09 17:40:07.604926 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.604905 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fwnfj" event={"ID":"0f515f8f-2a9d-40fc-8049-41438737d2bb","Type":"ContainerStarted","Data":"278615ddfe6c09d88c5897b15e89eda2d2f0eb7bb0c9db273cd547f30089f0d1"} Jul 09 17:40:07.606438 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.606346 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nzt4v" event={"ID":"f7e2b98f-c7e7-4796-b08e-820999127101","Type":"ContainerStarted","Data":"f0e944e81765d8b05fe872e2f46f3b829094a706d0d840a028e92dea66b6b93a"} Jul 09 17:40:07.609678 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.609659 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:40:07.612682 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.610904 2571 generic.go:356] "Generic (PLEG): container finished" podID="fede05a0-241f-47c0-aa3b-db1a5ccdbe63" containerID="2b3b301ebac65dbe2ecd3b3849da989428338bb7b5354cf1a3b7d3380e69a7a5" exitCode=1 Jul 09 17:40:07.612682 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.611322 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" event={"ID":"fede05a0-241f-47c0-aa3b-db1a5ccdbe63","Type":"ContainerStarted","Data":"c4519f17804d524a646d3e836af68091e1907ba462b0149505748244d76dd7f8"} Jul 09 17:40:07.612682 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.611411 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" event={"ID":"fede05a0-241f-47c0-aa3b-db1a5ccdbe63","Type":"ContainerStarted","Data":"7acdbe84840dc3514ed8859c04517262951cc7cf24208801c34d5011b3606469"} Jul 09 17:40:07.612682 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.611457 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" event={"ID":"fede05a0-241f-47c0-aa3b-db1a5ccdbe63","Type":"ContainerStarted","Data":"2f7665c99530622ce3c1209d6b830b7737b4e69b5ca05d957efeefdf5bea4c3c"} Jul 09 17:40:07.612682 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.611500 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" event={"ID":"fede05a0-241f-47c0-aa3b-db1a5ccdbe63","Type":"ContainerStarted","Data":"b039be60ffea396cada5d857d0c2965bd95bd0a503d47592b6aa2c28a0128b51"} Jul 09 17:40:07.612682 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.611542 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" event={"ID":"fede05a0-241f-47c0-aa3b-db1a5ccdbe63","Type":"ContainerDied","Data":"2b3b301ebac65dbe2ecd3b3849da989428338bb7b5354cf1a3b7d3380e69a7a5"} Jul 09 17:40:07.618459 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.618402 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 17:40:07.713051 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:07.713012 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fwnfj" podStartSLOduration=2.899368254 podStartE2EDuration="22.712999069s" totalImagesPullingTime="19.813630815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:07.710284864 +0000 UTC m=+23.796408340" watchObservedRunningTime="2026-07-09 17:40:07.712999069 +0000 UTC m=+23.799122534" Jul 09 17:40:08.364024 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:08.363919 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T17:40:07.61842526Z","UUID":"f9e708e6-31da-4b68-bf41-5e910fc90242","Handler":null,"Name":"","Endpoint":""} Jul 09 17:40:08.365859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:08.365836 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 17:40:08.366017 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:08.365878 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 17:40:08.472619 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:08.472588 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:08.472782 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:08.472732 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:40:08.615547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:08.615470 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fvp4m" event={"ID":"2b67de07-eb02-48c3-9e6e-31f3b83d24b7","Type":"ContainerStarted","Data":"f654bd11bfdf7248871a45b3b7118ccff713685b4ede433fbac3f60d7c49deb1"} Jul 09 17:40:08.617614 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:08.617576 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" event={"ID":"2d964e80-05f9-401a-ae9f-47e828370872","Type":"ContainerStarted","Data":"27be6cc7ad14f7c7093662cf983305a7789059979591cf582e1736795e9656d6"} Jul 09 17:40:08.634228 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:08.634188 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nzt4v" podStartSLOduration=4.045837913 podStartE2EDuration="23.634175254s" totalImagesPullingTime="19.588337341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:07.744834374 +0000 UTC m=+23.830957839" watchObservedRunningTime="2026-07-09 17:40:08.634175254 +0000 UTC m=+24.720298720" Jul 09 17:40:08.634359 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:08.634339 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fvp4m" podStartSLOduration=4.063456634 podStartE2EDuration="23.634334493s" totalImagesPullingTime="19.570877859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:08.63412264 +0000 UTC m=+24.720246106" watchObservedRunningTime="2026-07-09 17:40:08.634334493 +0000 UTC m=+24.720457958" Jul 09 17:40:08.882510 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:08.882287 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:40:08.882914 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:08.882887 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:40:09.472247 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:09.472173 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:09.472574 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:09.472170 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:09.472574 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:09.472307 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:09.472574 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:09.472370 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:40:09.623099 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:09.623074 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:40:09.623549 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:09.623518 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" event={"ID":"fede05a0-241f-47c0-aa3b-db1a5ccdbe63","Type":"ContainerStarted","Data":"9e64c54b13aa534d84f937e583f188802a45a0491735f97cf87f8f165139efbd"} Jul 09 17:40:09.625593 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:09.625565 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" event={"ID":"2d964e80-05f9-401a-ae9f-47e828370872","Type":"ContainerStarted","Data":"3cbb7ccb1c4181ee39fc3740f1c50eb804b90e9fc5304d0763c2b3abbd9c4788"} Jul 09 17:40:09.625854 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:09.625820 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:40:09.626366 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:09.626343 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rcl6f" Jul 09 17:40:09.707404 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:09.707358 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-td7n6" podStartSLOduration=7.500626603 podStartE2EDuration="24.7073415s" totalImagesPullingTime="17.206714897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:09.660437366 +0000 UTC m=+25.746560833" watchObservedRunningTime="2026-07-09 17:40:09.7073415 +0000 UTC m=+25.793464965" Jul 09 17:40:10.472515 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:10.472480 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:10.472678 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:10.472628 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:40:11.472604 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:11.472572 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:11.473173 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:11.472635 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:11.473173 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:11.472733 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:40:11.473173 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:11.472865 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:12.473056 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:12.473025 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:12.473635 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:12.473164 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:40:12.633844 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:12.633808 2571 generic.go:356] "Generic (PLEG): container finished" podID="1b0c760e-92c9-4e00-bf4e-bf578db74755" containerID="cee109d6c82fcb1794d14758f56e0f7f028a8d7f3b3a059dce4e1f4997a43f4b" exitCode=0 Jul 09 17:40:12.634014 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:12.633884 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px9wm" event={"ID":"1b0c760e-92c9-4e00-bf4e-bf578db74755","Type":"ContainerDied","Data":"cee109d6c82fcb1794d14758f56e0f7f028a8d7f3b3a059dce4e1f4997a43f4b"} Jul 09 17:40:12.637458 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:12.637440 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:40:12.637762 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:12.637744 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" event={"ID":"fede05a0-241f-47c0-aa3b-db1a5ccdbe63","Type":"ContainerStarted","Data":"16d39b69d5dcad53a1878dd9262fef4f162b43b3f997f72e7725db4e3a7405af"} Jul 09 17:40:12.638008 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:12.637995 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:40:12.638147 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:12.638134 2571 scope.go:117] "RemoveContainer" containerID="2b3b301ebac65dbe2ecd3b3849da989428338bb7b5354cf1a3b7d3380e69a7a5" Jul 09 17:40:12.653422 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:12.653404 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:40:13.244331 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.244305 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:40:13.473000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.472973 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:13.473000 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.472994 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:13.473594 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:13.473085 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:40:13.473594 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:13.473214 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:13.644701 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.644671 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:40:13.645190 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.645148 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" event={"ID":"fede05a0-241f-47c0-aa3b-db1a5ccdbe63","Type":"ContainerStarted","Data":"aa17f7cd879692bdfdb563a1e30f88247eaf76bc489e5dd535e9e14a49706882"} Jul 09 17:40:13.645834 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.645519 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:40:13.662292 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.662128 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:40:13.678345 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.678290 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" podStartSLOduration=8.92641663 podStartE2EDuration="28.678253105s" totalImagesPullingTime="19.751836475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:13.676365662 +0000 UTC m=+29.762489149" watchObservedRunningTime="2026-07-09 17:40:13.678253105 +0000 UTC m=+29.764376572" Jul 09 17:40:13.806706 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.806668 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2p7ff"] Jul 09 17:40:13.806841 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.806826 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:13.806959 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:13.806936 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:13.807524 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.807504 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pdswh"] Jul 09 17:40:13.807632 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.807594 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:13.807678 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:13.807661 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:40:13.822485 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.822464 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lwssb"] Jul 09 17:40:13.822574 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:13.822549 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:13.822642 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:13.822622 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:40:14.649158 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:14.649116 2571 generic.go:356] "Generic (PLEG): container finished" podID="1b0c760e-92c9-4e00-bf4e-bf578db74755" containerID="dc72e06cd586ed42734d23f1a1c6897265b487f7ebf7e90730f4870231b50f6d" exitCode=0 Jul 09 17:40:14.649723 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:14.649191 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px9wm" event={"ID":"1b0c760e-92c9-4e00-bf4e-bf578db74755","Type":"ContainerDied","Data":"dc72e06cd586ed42734d23f1a1c6897265b487f7ebf7e90730f4870231b50f6d"} Jul 09 17:40:15.472205 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:15.472171 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:15.472363 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:15.472294 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:15.472363 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:15.472304 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:15.472363 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:15.472303 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:15.472482 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:15.472372 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:40:15.472482 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:15.472435 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:40:16.655831 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:16.655798 2571 generic.go:356] "Generic (PLEG): container finished" podID="1b0c760e-92c9-4e00-bf4e-bf578db74755" containerID="87c301c6541618803f04f2bdae4b12e81226691411cac3f724caf9e054a70667" exitCode=0 Jul 09 17:40:16.656334 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:16.655851 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px9wm" event={"ID":"1b0c760e-92c9-4e00-bf4e-bf578db74755","Type":"ContainerDied","Data":"87c301c6541618803f04f2bdae4b12e81226691411cac3f724caf9e054a70667"} Jul 09 17:40:17.472031 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:17.471999 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:17.472213 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:17.471999 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:17.472213 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:17.472121 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2p7ff" podUID="c86cda2a-444c-431e-b014-9cef96fe3d8e" Jul 09 17:40:17.472362 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:17.472209 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdswh" podUID="5890960e-4225-4cae-83d2-a5749bb9d755" Jul 09 17:40:17.472362 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:17.472001 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:17.472362 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:17.472337 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwssb" podUID="02523ead-082e-41e1-ab84-648a74b97bd2" Jul 09 17:40:18.185238 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:18.185195 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:18.185692 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:18.185357 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:40:18.185692 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:18.185431 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret podName:5890960e-4225-4cae-83d2-a5749bb9d755 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:50.185408486 +0000 UTC m=+66.271531931 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret") pod "global-pull-secret-syncer-pdswh" (UID: "5890960e-4225-4cae-83d2-a5749bb9d755") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:40:18.185692 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:18.185471 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:18.185692 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:18.185563 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:40:18.185692 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:18.185626 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs podName:02523ead-082e-41e1-ab84-648a74b97bd2 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:50.18560684 +0000 UTC m=+66.271730290 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs") pod "network-metrics-daemon-lwssb" (UID: "02523ead-082e-41e1-ab84-648a74b97bd2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:40:18.286554 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:18.286523 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67pr\" (UniqueName: \"kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr\") pod \"network-check-target-2p7ff\" (UID: \"c86cda2a-444c-431e-b014-9cef96fe3d8e\") " pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:18.286703 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:18.286655 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:40:18.286703 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:18.286675 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:40:18.286703 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:18.286687 2571 projected.go:196] Error preparing data for projected volume kube-api-access-r67pr for pod openshift-network-diagnostics/network-check-target-2p7ff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:40:18.286856 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:18.286749 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr podName:c86cda2a-444c-431e-b014-9cef96fe3d8e nodeName:}" failed. No retries permitted until 2026-07-09 17:40:50.286731497 +0000 UTC m=+66.372854942 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r67pr" (UniqueName: "kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr") pod "network-check-target-2p7ff" (UID: "c86cda2a-444c-431e-b014-9cef96fe3d8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:40:19.225980 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.225695 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-63.ec2.internal" event="NodeReady" Jul 09 17:40:19.226432 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.226001 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 17:40:19.280391 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.280344 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7sfwq"] Jul 09 17:40:19.283113 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.283092 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lggb2"] Jul 09 17:40:19.283226 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.283216 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.285280 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.285247 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lggb2" Jul 09 17:40:19.286988 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.286810 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 17:40:19.286988 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.286864 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 17:40:19.286988 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.286875 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lljhl\"" Jul 09 17:40:19.287432 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.287412 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nnsbc\"" Jul 09 17:40:19.290021 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.290002 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 17:40:19.290102 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.290024 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 17:40:19.295125 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.295106 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lggb2"] Jul 09 17:40:19.295975 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.295931 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 17:40:19.302537 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.302519 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7sfwq"] Jul 09 17:40:19.394694 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.394654 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc75a7a-e966-43a2-a3e3-f946bd253e69-tmp-dir\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.394867 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.394750 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39bc423c-6220-41f9-a806-3208df828cbf-cert\") pod \"ingress-canary-lggb2\" (UID: \"39bc423c-6220-41f9-a806-3208df828cbf\") " pod="openshift-ingress-canary/ingress-canary-lggb2" Jul 09 17:40:19.394867 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.394775 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hjh7\" (UniqueName: \"kubernetes.io/projected/39bc423c-6220-41f9-a806-3208df828cbf-kube-api-access-6hjh7\") pod \"ingress-canary-lggb2\" (UID: \"39bc423c-6220-41f9-a806-3208df828cbf\") " pod="openshift-ingress-canary/ingress-canary-lggb2" Jul 09 17:40:19.394867 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.394821 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2bc75a7a-e966-43a2-a3e3-f946bd253e69-config-volume\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.395023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.394872 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.395023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.394902 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xhfw\" (UniqueName: \"kubernetes.io/projected/2bc75a7a-e966-43a2-a3e3-f946bd253e69-kube-api-access-6xhfw\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.472925 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.472890 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:19.472925 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.472911 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:19.473150 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.473006 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:19.479074 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.479013 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 17:40:19.479659 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.479639 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 17:40:19.479807 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.479701 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 17:40:19.480669 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.480642 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f99f5\"" Jul 09 17:40:19.481097 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.481078 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9rs28\"" Jul 09 17:40:19.482930 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.482911 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 17:40:19.495193 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.495165 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc75a7a-e966-43a2-a3e3-f946bd253e69-tmp-dir\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.495314 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.495245 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39bc423c-6220-41f9-a806-3208df828cbf-cert\") pod \"ingress-canary-lggb2\" (UID: \"39bc423c-6220-41f9-a806-3208df828cbf\") " pod="openshift-ingress-canary/ingress-canary-lggb2" Jul 09 17:40:19.495314 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.495290 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hjh7\" (UniqueName: \"kubernetes.io/projected/39bc423c-6220-41f9-a806-3208df828cbf-kube-api-access-6hjh7\") pod \"ingress-canary-lggb2\" (UID: \"39bc423c-6220-41f9-a806-3208df828cbf\") " pod="openshift-ingress-canary/ingress-canary-lggb2" Jul 09 17:40:19.495431 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.495314 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2bc75a7a-e966-43a2-a3e3-f946bd253e69-config-volume\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.495431 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.495349 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.495431 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.495375 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xhfw\" (UniqueName: \"kubernetes.io/projected/2bc75a7a-e966-43a2-a3e3-f946bd253e69-kube-api-access-6xhfw\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.495580 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.495563 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc75a7a-e966-43a2-a3e3-f946bd253e69-tmp-dir\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.495630 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:19.495619 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:40:19.495694 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:19.495679 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls podName:2bc75a7a-e966-43a2-a3e3-f946bd253e69 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:19.995659339 +0000 UTC m=+36.081782787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls") pod "dns-default-7sfwq" (UID: "2bc75a7a-e966-43a2-a3e3-f946bd253e69") : secret "dns-default-metrics-tls" not found Jul 09 17:40:19.496133 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.496111 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2bc75a7a-e966-43a2-a3e3-f946bd253e69-config-volume\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.500103 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.500083 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39bc423c-6220-41f9-a806-3208df828cbf-cert\") pod \"ingress-canary-lggb2\" (UID: \"39bc423c-6220-41f9-a806-3208df828cbf\") " pod="openshift-ingress-canary/ingress-canary-lggb2" Jul 09 17:40:19.507604 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.507582 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hjh7\" (UniqueName: \"kubernetes.io/projected/39bc423c-6220-41f9-a806-3208df828cbf-kube-api-access-6hjh7\") pod \"ingress-canary-lggb2\" (UID: \"39bc423c-6220-41f9-a806-3208df828cbf\") " pod="openshift-ingress-canary/ingress-canary-lggb2" Jul 09 17:40:19.507944 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.507924 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xhfw\" (UniqueName: \"kubernetes.io/projected/2bc75a7a-e966-43a2-a3e3-f946bd253e69-kube-api-access-6xhfw\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:19.603045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:19.603016 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lggb2" Jul 09 17:40:20.000551 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:20.000509 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:20.000753 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:20.000697 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:40:20.000808 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:20.000772 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls podName:2bc75a7a-e966-43a2-a3e3-f946bd253e69 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:21.000753475 +0000 UTC m=+37.086876939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls") pod "dns-default-7sfwq" (UID: "2bc75a7a-e966-43a2-a3e3-f946bd253e69") : secret "dns-default-metrics-tls" not found Jul 09 17:40:20.678413 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:20.678388 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lggb2"] Jul 09 17:40:20.682638 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:40:20.682601 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39bc423c_6220_41f9_a806_3208df828cbf.slice/crio-2a3b816f01556d896336a17abfdd5421b693ec320c14bf1e261cc1c58c706959 WatchSource:0}: Error finding container 2a3b816f01556d896336a17abfdd5421b693ec320c14bf1e261cc1c58c706959: Status 404 returned error can't find the container with id 2a3b816f01556d896336a17abfdd5421b693ec320c14bf1e261cc1c58c706959 Jul 09 17:40:21.008627 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:21.008447 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:21.008766 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:21.008600 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:40:21.008766 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:21.008723 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls podName:2bc75a7a-e966-43a2-a3e3-f946bd253e69 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:23.008708738 +0000 UTC m=+39.094832182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls") pod "dns-default-7sfwq" (UID: "2bc75a7a-e966-43a2-a3e3-f946bd253e69") : secret "dns-default-metrics-tls" not found Jul 09 17:40:21.670775 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:21.670740 2571 generic.go:356] "Generic (PLEG): container finished" podID="1b0c760e-92c9-4e00-bf4e-bf578db74755" containerID="500ed86671524caca4b973745492cad2109a97febbd8fe269838fad21ca24e25" exitCode=0 Jul 09 17:40:21.670971 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:21.670826 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px9wm" event={"ID":"1b0c760e-92c9-4e00-bf4e-bf578db74755","Type":"ContainerDied","Data":"500ed86671524caca4b973745492cad2109a97febbd8fe269838fad21ca24e25"} Jul 09 17:40:21.672430 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:21.672183 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lggb2" event={"ID":"39bc423c-6220-41f9-a806-3208df828cbf","Type":"ContainerStarted","Data":"2a3b816f01556d896336a17abfdd5421b693ec320c14bf1e261cc1c58c706959"} Jul 09 17:40:22.679201 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:22.679161 2571 generic.go:356] "Generic (PLEG): container finished" podID="1b0c760e-92c9-4e00-bf4e-bf578db74755" containerID="3a36eedd7b741791bb32d1edd79f26eaaa93f9c2abb2199183215621c7470c66" exitCode=0 Jul 09 17:40:22.679851 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:22.679228 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px9wm" event={"ID":"1b0c760e-92c9-4e00-bf4e-bf578db74755","Type":"ContainerDied","Data":"3a36eedd7b741791bb32d1edd79f26eaaa93f9c2abb2199183215621c7470c66"} Jul 09 17:40:23.024089 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:23.024066 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:23.024189 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:23.024177 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:40:23.024237 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:23.024229 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls podName:2bc75a7a-e966-43a2-a3e3-f946bd253e69 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:27.024215222 +0000 UTC m=+43.110338667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls") pod "dns-default-7sfwq" (UID: "2bc75a7a-e966-43a2-a3e3-f946bd253e69") : secret "dns-default-metrics-tls" not found Jul 09 17:40:23.684527 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:23.684435 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px9wm" event={"ID":"1b0c760e-92c9-4e00-bf4e-bf578db74755","Type":"ContainerStarted","Data":"22b236d3eb0721a624726b565c55911b386788de5103344db3bf1605aa9eaf60"} Jul 09 17:40:23.685845 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:23.685824 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lggb2" event={"ID":"39bc423c-6220-41f9-a806-3208df828cbf","Type":"ContainerStarted","Data":"3cd4946d2397489bd1451cf268b01bd8c81ff8547bc7cb8f4071b05517c8daa3"} Jul 09 17:40:23.712871 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:23.712827 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-px9wm" podStartSLOduration=8.276841383 podStartE2EDuration="38.712814828s" totalImagesPullingTime="30.435973445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:23.711539262 +0000 UTC m=+39.797662727" watchObservedRunningTime="2026-07-09 17:40:23.712814828 +0000 UTC m=+39.798938291" Jul 09 17:40:23.748353 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:23.748309 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lggb2" podStartSLOduration=2.483459609 podStartE2EDuration="4.748297233s" totalImagesPullingTime="2.264837624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:23.74819101 +0000 UTC m=+39.834314497" watchObservedRunningTime="2026-07-09 17:40:23.748297233 +0000 UTC m=+39.834420699" Jul 09 17:40:27.051464 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:27.051429 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:27.051931 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:27.051579 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:40:27.051931 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:27.051636 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls podName:2bc75a7a-e966-43a2-a3e3-f946bd253e69 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:35.051620249 +0000 UTC m=+51.137743696 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls") pod "dns-default-7sfwq" (UID: "2bc75a7a-e966-43a2-a3e3-f946bd253e69") : secret "dns-default-metrics-tls" not found Jul 09 17:40:35.109662 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:35.109628 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:35.110112 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:35.109734 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:40:35.110112 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:35.109795 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls podName:2bc75a7a-e966-43a2-a3e3-f946bd253e69 nodeName:}" failed. No retries permitted until 2026-07-09 17:40:51.10978061 +0000 UTC m=+67.195904054 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls") pod "dns-default-7sfwq" (UID: "2bc75a7a-e966-43a2-a3e3-f946bd253e69") : secret "dns-default-metrics-tls" not found Jul 09 17:40:45.661806 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:45.661778 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b547z" Jul 09 17:40:50.209494 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.209437 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:50.209947 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.209516 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:40:50.212414 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.212393 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 17:40:50.212801 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.212787 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 17:40:50.219733 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:50.219713 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 17:40:50.219788 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:50.219770 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs podName:02523ead-082e-41e1-ab84-648a74b97bd2 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:54.219755615 +0000 UTC m=+130.305879060 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs") pod "network-metrics-daemon-lwssb" (UID: "02523ead-082e-41e1-ab84-648a74b97bd2") : secret "metrics-daemon-secret" not found Jul 09 17:40:50.222924 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.222898 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5890960e-4225-4cae-83d2-a5749bb9d755-original-pull-secret\") pod \"global-pull-secret-syncer-pdswh\" (UID: \"5890960e-4225-4cae-83d2-a5749bb9d755\") " pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:50.310721 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.310663 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67pr\" (UniqueName: \"kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr\") pod \"network-check-target-2p7ff\" (UID: \"c86cda2a-444c-431e-b014-9cef96fe3d8e\") " pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:50.313430 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.313411 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 17:40:50.323938 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.323914 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 17:40:50.334703 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.334683 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r67pr\" (UniqueName: \"kubernetes.io/projected/c86cda2a-444c-431e-b014-9cef96fe3d8e-kube-api-access-r67pr\") pod \"network-check-target-2p7ff\" (UID: \"c86cda2a-444c-431e-b014-9cef96fe3d8e\") " pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:50.384766 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.384735 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdswh" Jul 09 17:40:50.400402 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.400383 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9rs28\"" Jul 09 17:40:50.408222 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.408203 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:50.540013 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.539981 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pdswh"] Jul 09 17:40:50.543627 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:40:50.543600 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5890960e_4225_4cae_83d2_a5749bb9d755.slice/crio-3bc4959117a5e054be84d66fd786dd310a116ac66a3af9d2578a776b9dc2674c WatchSource:0}: Error finding container 3bc4959117a5e054be84d66fd786dd310a116ac66a3af9d2578a776b9dc2674c: Status 404 returned error can't find the container with id 3bc4959117a5e054be84d66fd786dd310a116ac66a3af9d2578a776b9dc2674c Jul 09 17:40:50.553851 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.553824 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2p7ff"] Jul 09 17:40:50.558313 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:40:50.558290 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc86cda2a_444c_431e_b014_9cef96fe3d8e.slice/crio-71727a4d365cddfe63866bf4c81649ad59a289e2d7e6a2507dc6950c742cd9b0 WatchSource:0}: Error finding container 71727a4d365cddfe63866bf4c81649ad59a289e2d7e6a2507dc6950c742cd9b0: Status 404 returned error can't find the container with id 71727a4d365cddfe63866bf4c81649ad59a289e2d7e6a2507dc6950c742cd9b0 Jul 09 17:40:50.748047 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.747962 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2p7ff" event={"ID":"c86cda2a-444c-431e-b014-9cef96fe3d8e","Type":"ContainerStarted","Data":"71727a4d365cddfe63866bf4c81649ad59a289e2d7e6a2507dc6950c742cd9b0"} Jul 09 17:40:50.749092 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:50.749065 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pdswh" event={"ID":"5890960e-4225-4cae-83d2-a5749bb9d755","Type":"ContainerStarted","Data":"3bc4959117a5e054be84d66fd786dd310a116ac66a3af9d2578a776b9dc2674c"} Jul 09 17:40:51.116132 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:51.116095 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:40:51.116343 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:51.116219 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:40:51.116343 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:40:51.116309 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls podName:2bc75a7a-e966-43a2-a3e3-f946bd253e69 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:23.116293376 +0000 UTC m=+99.202416846 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls") pod "dns-default-7sfwq" (UID: "2bc75a7a-e966-43a2-a3e3-f946bd253e69") : secret "dns-default-metrics-tls" not found Jul 09 17:40:55.763564 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:55.763530 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2p7ff" event={"ID":"c86cda2a-444c-431e-b014-9cef96fe3d8e","Type":"ContainerStarted","Data":"ba77f1fe1b70de68f6fd98427d53c9339ab25ea46520c15abc9346c36722a691"} Jul 09 17:40:55.764085 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:55.763617 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:40:55.765022 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:55.764993 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pdswh" event={"ID":"5890960e-4225-4cae-83d2-a5749bb9d755","Type":"ContainerStarted","Data":"35e9dd6ff223d746998f6025aa43d131910621fe9202edee45d7b148be38a6b3"} Jul 09 17:40:55.779404 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:55.779355 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2p7ff" podStartSLOduration=66.249497048 podStartE2EDuration="1m10.77933991s" totalImagesPullingTime="4.529842862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:55.779155973 +0000 UTC m=+71.865279441" watchObservedRunningTime="2026-07-09 17:40:55.77933991 +0000 UTC m=+71.865463376" Jul 09 17:40:55.795073 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:40:55.795041 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pdswh" podStartSLOduration=66.248953518 podStartE2EDuration="1m10.795029951s" totalImagesPullingTime="4.546076433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:55.794667043 +0000 UTC m=+71.880790508" watchObservedRunningTime="2026-07-09 17:40:55.795029951 +0000 UTC m=+71.881153416" Jul 09 17:41:03.925626 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.925593 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx"] Jul 09 17:41:03.927443 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.927426 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx" Jul 09 17:41:03.935858 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.935838 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 17:41:03.938351 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.938337 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:41:03.941569 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.941553 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-q6h4t\"" Jul 09 17:41:03.972304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.972259 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj"] Jul 09 17:41:03.974488 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.974470 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx"] Jul 09 17:41:03.974560 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.974549 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:03.980872 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.980855 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 17:41:03.980872 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.980867 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 17:41:03.981020 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.980890 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 17:41:03.981846 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.981827 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 17:41:03.984950 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:03.984936 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-brkbp\"" Jul 09 17:41:04.003492 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.003474 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj"] Jul 09 17:41:04.008303 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.006108 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs944\" (UniqueName: \"kubernetes.io/projected/75bd3409-8739-41bc-88eb-501f310b7ecb-kube-api-access-rs944\") pod \"volume-data-source-validator-78495597f6-fkrwx\" (UID: \"75bd3409-8739-41bc-88eb-501f310b7ecb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx" Jul 09 17:41:04.106896 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.106857 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs944\" (UniqueName: \"kubernetes.io/projected/75bd3409-8739-41bc-88eb-501f310b7ecb-kube-api-access-rs944\") pod \"volume-data-source-validator-78495597f6-fkrwx\" (UID: \"75bd3409-8739-41bc-88eb-501f310b7ecb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx" Jul 09 17:41:04.106896 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.106896 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5e90ec89-3529-45c0-8422-fe0c83972cd4-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:04.107075 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.106931 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:04.107075 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.107014 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bcpf\" (UniqueName: \"kubernetes.io/projected/5e90ec89-3529-45c0-8422-fe0c83972cd4-kube-api-access-2bcpf\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:04.117775 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.117747 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs944\" (UniqueName: \"kubernetes.io/projected/75bd3409-8739-41bc-88eb-501f310b7ecb-kube-api-access-rs944\") pod \"volume-data-source-validator-78495597f6-fkrwx\" (UID: \"75bd3409-8739-41bc-88eb-501f310b7ecb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx" Jul 09 17:41:04.156341 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.156309 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-d79td"] Jul 09 17:41:04.158379 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.158362 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6699c6bb88-4kxjp"] Jul 09 17:41:04.158507 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.158489 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.160191 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.160172 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.162159 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.162144 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 09 17:41:04.167180 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.167160 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 09 17:41:04.167407 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.167384 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 17:41:04.167511 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.167450 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 17:41:04.167511 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.167478 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-5g4w2\"" Jul 09 17:41:04.167637 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.167560 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 09 17:41:04.167863 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.167844 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c6j4v\"" Jul 09 17:41:04.167958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.167905 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:41:04.167958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.167939 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 17:41:04.173222 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.173205 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 17:41:04.177619 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.177575 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 09 17:41:04.183820 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.183801 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6699c6bb88-4kxjp"] Jul 09 17:41:04.184520 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.184477 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-d79td"] Jul 09 17:41:04.207662 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.207634 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bcpf\" (UniqueName: \"kubernetes.io/projected/5e90ec89-3529-45c0-8422-fe0c83972cd4-kube-api-access-2bcpf\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:04.207752 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.207687 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5e90ec89-3529-45c0-8422-fe0c83972cd4-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:04.207752 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.207740 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:04.207864 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:04.207852 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:41:04.207939 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:04.207927 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls podName:5e90ec89-3529-45c0-8422-fe0c83972cd4 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:04.707907608 +0000 UTC m=+80.794031065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z75dj" (UID: "5e90ec89-3529-45c0-8422-fe0c83972cd4") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:41:04.208354 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.208337 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5e90ec89-3529-45c0-8422-fe0c83972cd4-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:04.222454 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.222435 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bcpf\" (UniqueName: \"kubernetes.io/projected/5e90ec89-3529-45c0-8422-fe0c83972cd4-kube-api-access-2bcpf\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:04.236254 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.236229 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309040 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-image-registry-private-configuration\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309088 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309117 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-serving-cert\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309152 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-ca-trust-extracted\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309211 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-trusted-ca\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309250 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mw8n\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-kube-api-access-7mw8n\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309298 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-trusted-ca\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309344 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98qs6\" (UniqueName: \"kubernetes.io/projected/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-kube-api-access-98qs6\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309363 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-config\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309401 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-certificates\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309439 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-bound-sa-token\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.309875 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.309471 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-installation-pull-secrets\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.353621 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.353593 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx"] Jul 09 17:41:04.356353 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:04.356327 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75bd3409_8739_41bc_88eb_501f310b7ecb.slice/crio-e0c3d60c0fb733130457517c8c042dfdfdcd1b35c9f2703061598fb926d65cb6 WatchSource:0}: Error finding container e0c3d60c0fb733130457517c8c042dfdfdcd1b35c9f2703061598fb926d65cb6: Status 404 returned error can't find the container with id e0c3d60c0fb733130457517c8c042dfdfdcd1b35c9f2703061598fb926d65cb6 Jul 09 17:41:04.410137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410110 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-image-registry-private-configuration\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.410137 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410136 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.410329 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410156 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-serving-cert\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.410329 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410176 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-ca-trust-extracted\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.410329 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410225 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-trusted-ca\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.410329 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:04.410259 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:41:04.410329 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410264 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mw8n\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-kube-api-access-7mw8n\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.410329 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:04.410292 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6699c6bb88-4kxjp: secret "image-registry-tls" not found Jul 09 17:41:04.410329 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410317 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-trusted-ca\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.410668 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:04.410349 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls podName:127cf5c0-21b8-4fb4-a147-a12d89d6ef7a nodeName:}" failed. No retries permitted until 2026-07-09 17:41:04.91032897 +0000 UTC m=+80.996452413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls") pod "image-registry-6699c6bb88-4kxjp" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a") : secret "image-registry-tls" not found Jul 09 17:41:04.410668 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410371 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98qs6\" (UniqueName: \"kubernetes.io/projected/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-kube-api-access-98qs6\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.410668 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410406 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-config\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.410668 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410441 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-certificates\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.410668 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410470 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-bound-sa-token\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.410668 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410502 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-installation-pull-secrets\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.410958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.410675 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-ca-trust-extracted\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.411155 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.411019 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-certificates\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.411289 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.411254 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-trusted-ca\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.411454 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.411384 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-trusted-ca\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.411626 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.411598 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-config\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.412716 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.412694 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-image-registry-private-configuration\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.416165 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.413442 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-installation-pull-secrets\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.416165 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.413448 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-serving-cert\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.419619 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.419592 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-bound-sa-token\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.419713 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.419689 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98qs6\" (UniqueName: \"kubernetes.io/projected/04a9b6e5-8ddf-4e6f-ad75-e746722fcb12-kube-api-access-98qs6\") pod \"console-operator-7866497494-d79td\" (UID: \"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12\") " pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.419762 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.419740 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mw8n\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-kube-api-access-7mw8n\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.469726 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.469663 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:04.586653 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.586624 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-d79td"] Jul 09 17:41:04.590181 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:04.590151 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a9b6e5_8ddf_4e6f_ad75_e746722fcb12.slice/crio-3858417dabbeb853d0ac0cdd767697862888ca85328ae983cf28c0a6b5a0468a WatchSource:0}: Error finding container 3858417dabbeb853d0ac0cdd767697862888ca85328ae983cf28c0a6b5a0468a: Status 404 returned error can't find the container with id 3858417dabbeb853d0ac0cdd767697862888ca85328ae983cf28c0a6b5a0468a Jul 09 17:41:04.713117 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.713087 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:04.713307 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:04.713212 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:41:04.713307 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:04.713280 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls podName:5e90ec89-3529-45c0-8422-fe0c83972cd4 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:05.713252401 +0000 UTC m=+81.799375845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z75dj" (UID: "5e90ec89-3529-45c0-8422-fe0c83972cd4") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:41:04.790342 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.790247 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-d79td" event={"ID":"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12","Type":"ContainerStarted","Data":"3858417dabbeb853d0ac0cdd767697862888ca85328ae983cf28c0a6b5a0468a"} Jul 09 17:41:04.791489 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.791463 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx" event={"ID":"75bd3409-8739-41bc-88eb-501f310b7ecb","Type":"ContainerStarted","Data":"e0c3d60c0fb733130457517c8c042dfdfdcd1b35c9f2703061598fb926d65cb6"} Jul 09 17:41:04.914929 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:04.914895 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:04.915068 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:04.915007 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:41:04.915068 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:04.915018 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6699c6bb88-4kxjp: secret "image-registry-tls" not found Jul 09 17:41:04.915068 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:04.915064 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls podName:127cf5c0-21b8-4fb4-a147-a12d89d6ef7a nodeName:}" failed. No retries permitted until 2026-07-09 17:41:05.915050874 +0000 UTC m=+82.001174318 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls") pod "image-registry-6699c6bb88-4kxjp" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a") : secret "image-registry-tls" not found Jul 09 17:41:05.722206 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:05.722121 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:05.722567 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:05.722291 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:41:05.722567 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:05.722362 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls podName:5e90ec89-3529-45c0-8422-fe0c83972cd4 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:07.722342723 +0000 UTC m=+83.808466170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z75dj" (UID: "5e90ec89-3529-45c0-8422-fe0c83972cd4") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:41:05.794890 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:05.794853 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx" event={"ID":"75bd3409-8739-41bc-88eb-501f310b7ecb","Type":"ContainerStarted","Data":"c9f2f83feb95051fb65fb6de2a112d95417cbc903e84451a27c7aea89e4edd11"} Jul 09 17:41:05.812637 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:05.812575 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-fkrwx" podStartSLOduration=1.732775481 podStartE2EDuration="2.812554009s" totalImagesPullingTime="1.079778528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:05.811515819 +0000 UTC m=+81.897639306" watchObservedRunningTime="2026-07-09 17:41:05.812554009 +0000 UTC m=+81.898677516" Jul 09 17:41:05.924085 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:05.924048 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:05.924233 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:05.924207 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:41:05.924233 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:05.924230 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6699c6bb88-4kxjp: secret "image-registry-tls" not found Jul 09 17:41:05.924337 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:05.924307 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls podName:127cf5c0-21b8-4fb4-a147-a12d89d6ef7a nodeName:}" failed. No retries permitted until 2026-07-09 17:41:07.924286572 +0000 UTC m=+84.010410033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls") pod "image-registry-6699c6bb88-4kxjp" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a") : secret "image-registry-tls" not found Jul 09 17:41:06.800211 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:06.800188 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/0.log" Jul 09 17:41:06.800648 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:06.800227 2571 generic.go:356] "Generic (PLEG): container finished" podID="04a9b6e5-8ddf-4e6f-ad75-e746722fcb12" containerID="c42b8e55729a9bf8b0f2dee62c42b6ecb22351bc830a638aea3a16021e761ad3" exitCode=255 Jul 09 17:41:06.800648 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:06.800303 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-d79td" event={"ID":"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12","Type":"ContainerDied","Data":"c42b8e55729a9bf8b0f2dee62c42b6ecb22351bc830a638aea3a16021e761ad3"} Jul 09 17:41:06.800648 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:06.800569 2571 scope.go:117] "RemoveContainer" containerID="c42b8e55729a9bf8b0f2dee62c42b6ecb22351bc830a638aea3a16021e761ad3" Jul 09 17:41:07.738680 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:07.738646 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:07.738877 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:07.738809 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:41:07.738951 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:07.738887 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls podName:5e90ec89-3529-45c0-8422-fe0c83972cd4 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:11.738867402 +0000 UTC m=+87.824990848 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z75dj" (UID: "5e90ec89-3529-45c0-8422-fe0c83972cd4") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:41:07.807212 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:07.807186 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:41:07.807657 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:07.807640 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/0.log" Jul 09 17:41:07.807722 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:07.807681 2571 generic.go:356] "Generic (PLEG): container finished" podID="04a9b6e5-8ddf-4e6f-ad75-e746722fcb12" containerID="2176af5fb95006e319200edaefe504af90234d087eeac27907b1efd7623dd946" exitCode=255 Jul 09 17:41:07.807776 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:07.807731 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-d79td" event={"ID":"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12","Type":"ContainerDied","Data":"2176af5fb95006e319200edaefe504af90234d087eeac27907b1efd7623dd946"} Jul 09 17:41:07.807832 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:07.807820 2571 scope.go:117] "RemoveContainer" containerID="c42b8e55729a9bf8b0f2dee62c42b6ecb22351bc830a638aea3a16021e761ad3" Jul 09 17:41:07.808007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:07.807989 2571 scope.go:117] "RemoveContainer" containerID="2176af5fb95006e319200edaefe504af90234d087eeac27907b1efd7623dd946" Jul 09 17:41:07.808174 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:07.808152 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-d79td_openshift-console-operator(04a9b6e5-8ddf-4e6f-ad75-e746722fcb12)\"" pod="openshift-console-operator/console-operator-7866497494-d79td" podUID="04a9b6e5-8ddf-4e6f-ad75-e746722fcb12" Jul 09 17:41:07.939670 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:07.939635 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:07.939909 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:07.939887 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:41:07.939956 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:07.939910 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6699c6bb88-4kxjp: secret "image-registry-tls" not found Jul 09 17:41:07.939990 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:07.939965 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls podName:127cf5c0-21b8-4fb4-a147-a12d89d6ef7a nodeName:}" failed. No retries permitted until 2026-07-09 17:41:11.939946364 +0000 UTC m=+88.026069808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls") pod "image-registry-6699c6bb88-4kxjp" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a") : secret "image-registry-tls" not found Jul 09 17:41:08.027887 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.027809 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qxfks"] Jul 09 17:41:08.030701 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.030681 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.034077 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.034035 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 17:41:08.034446 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.034430 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n86q2\"" Jul 09 17:41:08.034632 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.034620 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 17:41:08.034885 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.034871 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 17:41:08.035594 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.035565 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 17:41:08.047816 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.047792 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qxfks"] Jul 09 17:41:08.141572 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.141534 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.141721 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.141584 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/193ce01b-0b42-4397-9168-2848c5d8fb92-crio-socket\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.141721 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.141616 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.141721 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.141679 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns7jw\" (UniqueName: \"kubernetes.io/projected/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-api-access-ns7jw\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.141721 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.141706 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/193ce01b-0b42-4397-9168-2848c5d8fb92-data-volume\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.242822 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.242787 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.242981 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.242841 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/193ce01b-0b42-4397-9168-2848c5d8fb92-crio-socket\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.242981 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.242872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.242981 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:08.242905 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:41:08.243127 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.242976 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/193ce01b-0b42-4397-9168-2848c5d8fb92-crio-socket\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.243127 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:08.242983 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:41:08.243127 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.242910 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns7jw\" (UniqueName: \"kubernetes.io/projected/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-api-access-ns7jw\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.243127 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:08.242986 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls podName:193ce01b-0b42-4397-9168-2848c5d8fb92 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:08.742969131 +0000 UTC m=+84.829092575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qxfks" (UID: "193ce01b-0b42-4397-9168-2848c5d8fb92") : secret "insights-runtime-extractor-tls" not found Jul 09 17:41:08.243127 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:08.243100 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm podName:193ce01b-0b42-4397-9168-2848c5d8fb92 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:08.743088302 +0000 UTC m=+84.829211746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qxfks" (UID: "193ce01b-0b42-4397-9168-2848c5d8fb92") : configmap "kube-rbac-proxy" not found Jul 09 17:41:08.243127 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.243122 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/193ce01b-0b42-4397-9168-2848c5d8fb92-data-volume\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.243466 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.243451 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/193ce01b-0b42-4397-9168-2848c5d8fb92-data-volume\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.257717 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.257693 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns7jw\" (UniqueName: \"kubernetes.io/projected/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-api-access-ns7jw\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.746830 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.746797 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.747019 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.746850 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:08.747019 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:08.746948 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:41:08.747019 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:08.747010 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls podName:193ce01b-0b42-4397-9168-2848c5d8fb92 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:09.746993259 +0000 UTC m=+85.833116702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qxfks" (UID: "193ce01b-0b42-4397-9168-2848c5d8fb92") : secret "insights-runtime-extractor-tls" not found Jul 09 17:41:08.747172 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:08.746949 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:41:08.747172 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:08.747100 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm podName:193ce01b-0b42-4397-9168-2848c5d8fb92 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:09.747083317 +0000 UTC m=+85.833206774 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qxfks" (UID: "193ce01b-0b42-4397-9168-2848c5d8fb92") : configmap "kube-rbac-proxy" not found Jul 09 17:41:08.812499 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.812471 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:41:08.812820 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:08.812778 2571 scope.go:117] "RemoveContainer" containerID="2176af5fb95006e319200edaefe504af90234d087eeac27907b1efd7623dd946" Jul 09 17:41:08.812946 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:08.812926 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-d79td_openshift-console-operator(04a9b6e5-8ddf-4e6f-ad75-e746722fcb12)\"" pod="openshift-console-operator/console-operator-7866497494-d79td" podUID="04a9b6e5-8ddf-4e6f-ad75-e746722fcb12" Jul 09 17:41:09.754758 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:09.754719 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:09.754758 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:09.754767 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:09.754981 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:09.754864 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:41:09.754981 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:09.754878 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:41:09.754981 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:09.754929 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls podName:193ce01b-0b42-4397-9168-2848c5d8fb92 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:11.75491265 +0000 UTC m=+87.841036097 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qxfks" (UID: "193ce01b-0b42-4397-9168-2848c5d8fb92") : secret "insights-runtime-extractor-tls" not found Jul 09 17:41:09.754981 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:09.754944 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm podName:193ce01b-0b42-4397-9168-2848c5d8fb92 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:11.754937928 +0000 UTC m=+87.841061372 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qxfks" (UID: "193ce01b-0b42-4397-9168-2848c5d8fb92") : configmap "kube-rbac-proxy" not found Jul 09 17:41:10.195578 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:10.195553 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nzt4v_f7e2b98f-c7e7-4796-b08e-820999127101/dns-node-resolver/0.log" Jul 09 17:41:11.196553 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:11.196527 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-czxd6_1654e8a4-0a47-4d4f-a3de-35a68c06bcea/node-ca/0.log" Jul 09 17:41:11.771053 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:11.771017 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:11.771207 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:11.771073 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:11.771207 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:11.771112 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:11.771207 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:11.771172 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:41:11.771207 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:11.771198 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:41:11.771364 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:11.771226 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:41:11.771364 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:11.771242 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls podName:5e90ec89-3529-45c0-8422-fe0c83972cd4 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:19.771223493 +0000 UTC m=+95.857346952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z75dj" (UID: "5e90ec89-3529-45c0-8422-fe0c83972cd4") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:41:11.771364 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:11.771256 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm podName:193ce01b-0b42-4397-9168-2848c5d8fb92 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:15.77125015 +0000 UTC m=+91.857373594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qxfks" (UID: "193ce01b-0b42-4397-9168-2848c5d8fb92") : configmap "kube-rbac-proxy" not found Jul 09 17:41:11.771364 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:11.771295 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls podName:193ce01b-0b42-4397-9168-2848c5d8fb92 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:15.771262472 +0000 UTC m=+91.857385916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qxfks" (UID: "193ce01b-0b42-4397-9168-2848c5d8fb92") : secret "insights-runtime-extractor-tls" not found Jul 09 17:41:11.795325 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:11.795305 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lggb2_39bc423c-6220-41f9-a806-3208df828cbf/serve-healthcheck-canary/0.log" Jul 09 17:41:11.972974 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:11.972944 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:11.973115 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:11.973086 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:41:11.973115 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:11.973100 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6699c6bb88-4kxjp: secret "image-registry-tls" not found Jul 09 17:41:11.973179 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:11.973160 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls podName:127cf5c0-21b8-4fb4-a147-a12d89d6ef7a nodeName:}" failed. No retries permitted until 2026-07-09 17:41:19.973135711 +0000 UTC m=+96.059259164 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls") pod "image-registry-6699c6bb88-4kxjp" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a") : secret "image-registry-tls" not found Jul 09 17:41:14.470596 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:14.470552 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:14.470941 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:14.470611 2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:14.470941 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:14.470902 2571 scope.go:117] "RemoveContainer" containerID="2176af5fb95006e319200edaefe504af90234d087eeac27907b1efd7623dd946" Jul 09 17:41:14.471060 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:14.471038 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-d79td_openshift-console-operator(04a9b6e5-8ddf-4e6f-ad75-e746722fcb12)\"" pod="openshift-console-operator/console-operator-7866497494-d79td" podUID="04a9b6e5-8ddf-4e6f-ad75-e746722fcb12" Jul 09 17:41:15.803993 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:15.803961 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:15.804416 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:15.804009 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:15.804416 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:15.804103 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:41:15.804416 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:15.804158 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls podName:193ce01b-0b42-4397-9168-2848c5d8fb92 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:23.804142079 +0000 UTC m=+99.890265522 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qxfks" (UID: "193ce01b-0b42-4397-9168-2848c5d8fb92") : secret "insights-runtime-extractor-tls" not found Jul 09 17:41:15.804416 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:15.804188 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:41:15.804416 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:15.804240 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm podName:193ce01b-0b42-4397-9168-2848c5d8fb92 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:23.804224156 +0000 UTC m=+99.890347601 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qxfks" (UID: "193ce01b-0b42-4397-9168-2848c5d8fb92") : configmap "kube-rbac-proxy" not found Jul 09 17:41:19.834919 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:19.834877 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:19.837338 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:19.837312 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e90ec89-3529-45c0-8422-fe0c83972cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z75dj\" (UID: \"5e90ec89-3529-45c0-8422-fe0c83972cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:19.882116 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:19.882079 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" Jul 09 17:41:19.999692 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:19.999548 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj"] Jul 09 17:41:20.002398 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:20.002367 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e90ec89_3529_45c0_8422_fe0c83972cd4.slice/crio-4fab703d230809283483095bce1ad8eb7901b6f3f3a51b997aa13912341b7efa WatchSource:0}: Error finding container 4fab703d230809283483095bce1ad8eb7901b6f3f3a51b997aa13912341b7efa: Status 404 returned error can't find the container with id 4fab703d230809283483095bce1ad8eb7901b6f3f3a51b997aa13912341b7efa Jul 09 17:41:20.036584 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:20.036555 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:20.038851 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:20.038828 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls\") pod \"image-registry-6699c6bb88-4kxjp\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:20.073910 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:20.073888 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:20.201834 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:20.201798 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6699c6bb88-4kxjp"] Jul 09 17:41:20.204530 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:20.204497 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod127cf5c0_21b8_4fb4_a147_a12d89d6ef7a.slice/crio-3e8863c1098a273f426ba8029a13795997660f9f29b972c58d659897b9c32a5b WatchSource:0}: Error finding container 3e8863c1098a273f426ba8029a13795997660f9f29b972c58d659897b9c32a5b: Status 404 returned error can't find the container with id 3e8863c1098a273f426ba8029a13795997660f9f29b972c58d659897b9c32a5b Jul 09 17:41:20.850583 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:20.850546 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" event={"ID":"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a","Type":"ContainerStarted","Data":"b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1"} Jul 09 17:41:20.851001 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:20.850600 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" event={"ID":"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a","Type":"ContainerStarted","Data":"3e8863c1098a273f426ba8029a13795997660f9f29b972c58d659897b9c32a5b"} Jul 09 17:41:20.851001 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:20.850640 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:20.851917 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:20.851890 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" event={"ID":"5e90ec89-3529-45c0-8422-fe0c83972cd4","Type":"ContainerStarted","Data":"4fab703d230809283483095bce1ad8eb7901b6f3f3a51b997aa13912341b7efa"} Jul 09 17:41:20.871525 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:20.871474 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" podStartSLOduration=16.871460876 podStartE2EDuration="16.871460876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:20.870479214 +0000 UTC m=+96.956602706" watchObservedRunningTime="2026-07-09 17:41:20.871460876 +0000 UTC m=+96.957584342" Jul 09 17:41:21.857327 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:21.857254 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" event={"ID":"5e90ec89-3529-45c0-8422-fe0c83972cd4","Type":"ContainerStarted","Data":"5cbdc37650135e7f286342e4117379eeca36f6ed5bade6f6f33c29c8d1b1bd51"} Jul 09 17:41:21.874631 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:21.874576 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z75dj" podStartSLOduration=17.158000314 podStartE2EDuration="18.874556581s" totalImagesPullingTime="1.716556267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:21.873751457 +0000 UTC m=+97.959874924" watchObservedRunningTime="2026-07-09 17:41:21.874556581 +0000 UTC m=+97.960680047" Jul 09 17:41:23.155914 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.155879 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:41:23.158127 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.158103 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc75a7a-e966-43a2-a3e3-f946bd253e69-metrics-tls\") pod \"dns-default-7sfwq\" (UID: \"2bc75a7a-e966-43a2-a3e3-f946bd253e69\") " pod="openshift-dns/dns-default-7sfwq" Jul 09 17:41:23.198903 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.198878 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lljhl\"" Jul 09 17:41:23.207111 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.207093 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7sfwq" Jul 09 17:41:23.331032 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.331002 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7sfwq"] Jul 09 17:41:23.333725 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:23.333697 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc75a7a_e966_43a2_a3e3_f946bd253e69.slice/crio-b08c1c27656da1c6741f328b6f996b5f19b2b43e6083be51b18664425450efe6 WatchSource:0}: Error finding container b08c1c27656da1c6741f328b6f996b5f19b2b43e6083be51b18664425450efe6: Status 404 returned error can't find the container with id b08c1c27656da1c6741f328b6f996b5f19b2b43e6083be51b18664425450efe6 Jul 09 17:41:23.861335 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.861304 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:23.861491 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.861343 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:23.861796 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.861775 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/193ce01b-0b42-4397-9168-2848c5d8fb92-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:23.863594 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.863570 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/193ce01b-0b42-4397-9168-2848c5d8fb92-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qxfks\" (UID: \"193ce01b-0b42-4397-9168-2848c5d8fb92\") " pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:23.864701 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.864678 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7sfwq" event={"ID":"2bc75a7a-e966-43a2-a3e3-f946bd253e69","Type":"ContainerStarted","Data":"b08c1c27656da1c6741f328b6f996b5f19b2b43e6083be51b18664425450efe6"} Jul 09 17:41:23.939209 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:23.939176 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qxfks" Jul 09 17:41:24.070870 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:24.070841 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qxfks"] Jul 09 17:41:24.073402 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:24.073376 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod193ce01b_0b42_4397_9168_2848c5d8fb92.slice/crio-dc91f10835e2fc93a6b3f827de46901b0843a3a9f3eba0408c3d93f256f6c2d6 WatchSource:0}: Error finding container dc91f10835e2fc93a6b3f827de46901b0843a3a9f3eba0408c3d93f256f6c2d6: Status 404 returned error can't find the container with id dc91f10835e2fc93a6b3f827de46901b0843a3a9f3eba0408c3d93f256f6c2d6 Jul 09 17:41:24.874256 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:24.874209 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qxfks" event={"ID":"193ce01b-0b42-4397-9168-2848c5d8fb92","Type":"ContainerStarted","Data":"dc91f10835e2fc93a6b3f827de46901b0843a3a9f3eba0408c3d93f256f6c2d6"} Jul 09 17:41:26.472405 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:26.472374 2571 scope.go:117] "RemoveContainer" containerID="2176af5fb95006e319200edaefe504af90234d087eeac27907b1efd7623dd946" Jul 09 17:41:26.770523 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:26.770454 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2p7ff" Jul 09 17:41:28.888084 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:28.888057 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:41:28.888488 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:28.888159 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-d79td" event={"ID":"04a9b6e5-8ddf-4e6f-ad75-e746722fcb12","Type":"ContainerStarted","Data":"d0f5ad1ffc1efe670f81f2bd68ea71b54cb3d67bf3222045e2b6e9f41430bee1"} Jul 09 17:41:28.888488 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:28.888472 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:28.890317 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:28.890291 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7sfwq" event={"ID":"2bc75a7a-e966-43a2-a3e3-f946bd253e69","Type":"ContainerStarted","Data":"71c8e18de2263f1c21a981d356c566e6c596cfd4b0d23a350665f8edebb03134"} Jul 09 17:41:28.890444 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:28.890332 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7sfwq" event={"ID":"2bc75a7a-e966-43a2-a3e3-f946bd253e69","Type":"ContainerStarted","Data":"34b242e4718eca0a18bf7753c6d3fc8d8394f0837253159352ed234bfd92c165"} Jul 09 17:41:28.890444 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:28.890352 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7sfwq" Jul 09 17:41:28.891760 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:28.891739 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qxfks" event={"ID":"193ce01b-0b42-4397-9168-2848c5d8fb92","Type":"ContainerStarted","Data":"16b8bdd03bd7ff6239942ca3661b668932d48b34ee4424ceeaefdcfa95398a01"} Jul 09 17:41:28.909058 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:28.908981 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-d79td" podStartSLOduration=22.866680242 podStartE2EDuration="24.908967934s" totalImagesPullingTime="2.042287692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:28.907902601 +0000 UTC m=+104.994026106" watchObservedRunningTime="2026-07-09 17:41:28.908967934 +0000 UTC m=+104.995091400" Jul 09 17:41:28.929211 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:28.929169 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7sfwq" podStartSLOduration=64.608229244 podStartE2EDuration="1m9.929158069s" totalImagesPullingTime="5.320928825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:28.929011335 +0000 UTC m=+105.015134802" watchObservedRunningTime="2026-07-09 17:41:28.929158069 +0000 UTC m=+105.015281534" Jul 09 17:41:29.091384 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.091353 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-d79td" Jul 09 17:41:29.424682 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.424657 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz"] Jul 09 17:41:29.426532 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.426515 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" Jul 09 17:41:29.429037 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.429021 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 17:41:29.429129 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.429118 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 17:41:29.429448 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.429425 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 17:41:29.430214 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.430190 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 09 17:41:29.430302 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.430227 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6g89j\"" Jul 09 17:41:29.432705 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.432617 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6699c6bb88-4kxjp"] Jul 09 17:41:29.439243 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.439222 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz"] Jul 09 17:41:29.446682 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.446663 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l"] Jul 09 17:41:29.448519 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.448495 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l" Jul 09 17:41:29.451685 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.451666 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bx8f7\"" Jul 09 17:41:29.455653 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.455631 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 09 17:41:29.483303 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.483261 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l"] Jul 09 17:41:29.608618 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.608540 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf26x\" (UniqueName: \"kubernetes.io/projected/4e0be933-6aac-4a1b-964e-f43b1ec01455-kube-api-access-cf26x\") pod \"managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz\" (UID: \"4e0be933-6aac-4a1b-964e-f43b1ec01455\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" Jul 09 17:41:29.608618 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.608581 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e0be933-6aac-4a1b-964e-f43b1ec01455-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz\" (UID: \"4e0be933-6aac-4a1b-964e-f43b1ec01455\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" Jul 09 17:41:29.608618 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.608606 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/813a8dc1-6c10-444d-8dda-69d2c32c9acc-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6l54l\" (UID: \"813a8dc1-6c10-444d-8dda-69d2c32c9acc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l" Jul 09 17:41:29.709803 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.709772 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf26x\" (UniqueName: \"kubernetes.io/projected/4e0be933-6aac-4a1b-964e-f43b1ec01455-kube-api-access-cf26x\") pod \"managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz\" (UID: \"4e0be933-6aac-4a1b-964e-f43b1ec01455\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" Jul 09 17:41:29.709803 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.709806 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e0be933-6aac-4a1b-964e-f43b1ec01455-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz\" (UID: \"4e0be933-6aac-4a1b-964e-f43b1ec01455\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" Jul 09 17:41:29.709973 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.709828 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/813a8dc1-6c10-444d-8dda-69d2c32c9acc-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6l54l\" (UID: \"813a8dc1-6c10-444d-8dda-69d2c32c9acc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l" Jul 09 17:41:29.712204 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.712181 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/813a8dc1-6c10-444d-8dda-69d2c32c9acc-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6l54l\" (UID: \"813a8dc1-6c10-444d-8dda-69d2c32c9acc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l" Jul 09 17:41:29.712329 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.712298 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e0be933-6aac-4a1b-964e-f43b1ec01455-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz\" (UID: \"4e0be933-6aac-4a1b-964e-f43b1ec01455\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" Jul 09 17:41:29.721801 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.721778 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf26x\" (UniqueName: \"kubernetes.io/projected/4e0be933-6aac-4a1b-964e-f43b1ec01455-kube-api-access-cf26x\") pod \"managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz\" (UID: \"4e0be933-6aac-4a1b-964e-f43b1ec01455\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" Jul 09 17:41:29.749208 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.749186 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" Jul 09 17:41:29.757828 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.757708 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l" Jul 09 17:41:29.894781 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.894259 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz"] Jul 09 17:41:29.897445 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:29.897417 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e0be933_6aac_4a1b_964e_f43b1ec01455.slice/crio-a416089aac0ec0ce4e995cfb1357786ead755b0d062b135e329802dab47e9118 WatchSource:0}: Error finding container a416089aac0ec0ce4e995cfb1357786ead755b0d062b135e329802dab47e9118: Status 404 returned error can't find the container with id a416089aac0ec0ce4e995cfb1357786ead755b0d062b135e329802dab47e9118 Jul 09 17:41:29.898891 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.898859 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qxfks" event={"ID":"193ce01b-0b42-4397-9168-2848c5d8fb92","Type":"ContainerStarted","Data":"c4506e1c74e8addffe86c4628161e15c1b777f859294828ad65b9a72586863dc"} Jul 09 17:41:29.916944 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:29.916919 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l"] Jul 09 17:41:30.905226 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:30.905174 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l" event={"ID":"813a8dc1-6c10-444d-8dda-69d2c32c9acc","Type":"ContainerStarted","Data":"50e495b9a4cd92f68876641c3ca6bfc02196cc2c8c70b71a505943aa69b40745"} Jul 09 17:41:30.906480 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:30.906447 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" event={"ID":"4e0be933-6aac-4a1b-964e-f43b1ec01455","Type":"ContainerStarted","Data":"a416089aac0ec0ce4e995cfb1357786ead755b0d062b135e329802dab47e9118"} Jul 09 17:41:32.916178 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:32.916143 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qxfks" event={"ID":"193ce01b-0b42-4397-9168-2848c5d8fb92","Type":"ContainerStarted","Data":"e09d5ceedaabcc597a0f809f15f7237a9e0989188edaee2d53040624dd097282"} Jul 09 17:41:32.917639 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:32.917619 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l" event={"ID":"813a8dc1-6c10-444d-8dda-69d2c32c9acc","Type":"ContainerStarted","Data":"2ffe2543499c0809ba39ff9685a4a3c806a40ea05e12e5946874cc97746a2676"} Jul 09 17:41:32.917818 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:32.917805 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l" Jul 09 17:41:32.922433 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:32.922412 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l" Jul 09 17:41:32.933454 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:32.933406 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qxfks" podStartSLOduration=17.986765559 podStartE2EDuration="25.933391025s" totalImagesPullingTime="7.946625466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:41:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:32.933044409 +0000 UTC m=+109.019167874" watchObservedRunningTime="2026-07-09 17:41:32.933391025 +0000 UTC m=+109.019514490" Jul 09 17:41:32.948251 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:32.948214 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6l54l" podStartSLOduration=1.741981493 podStartE2EDuration="3.948199834s" totalImagesPullingTime="2.206218341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:32.947726344 +0000 UTC m=+109.033849837" watchObservedRunningTime="2026-07-09 17:41:32.948199834 +0000 UTC m=+109.034323300" Jul 09 17:41:33.273810 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.273737 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-sqwxw"] Jul 09 17:41:33.276147 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.276130 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.279553 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.279518 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 09 17:41:33.279700 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.279647 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-kmwv5\"" Jul 09 17:41:33.280565 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.280548 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 17:41:33.280683 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.280575 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 09 17:41:33.286589 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.286566 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-sqwxw"] Jul 09 17:41:33.332552 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.332525 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnwd9\" (UniqueName: \"kubernetes.io/projected/58e45f2e-a384-481f-b163-fa4accf8c0a1-kube-api-access-nnwd9\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.332673 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.332584 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/58e45f2e-a384-481f-b163-fa4accf8c0a1-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.332673 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.332602 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58e45f2e-a384-481f-b163-fa4accf8c0a1-metrics-client-ca\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.332673 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.332660 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58e45f2e-a384-481f-b163-fa4accf8c0a1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.432996 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.432967 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/58e45f2e-a384-481f-b163-fa4accf8c0a1-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.432996 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.432999 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58e45f2e-a384-481f-b163-fa4accf8c0a1-metrics-client-ca\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.433143 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.433029 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58e45f2e-a384-481f-b163-fa4accf8c0a1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.433143 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.433065 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnwd9\" (UniqueName: \"kubernetes.io/projected/58e45f2e-a384-481f-b163-fa4accf8c0a1-kube-api-access-nnwd9\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.433143 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:33.433107 2571 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 09 17:41:33.433242 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:33.433166 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58e45f2e-a384-481f-b163-fa4accf8c0a1-prometheus-operator-tls podName:58e45f2e-a384-481f-b163-fa4accf8c0a1 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:33.933145769 +0000 UTC m=+110.019269215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/58e45f2e-a384-481f-b163-fa4accf8c0a1-prometheus-operator-tls") pod "prometheus-operator-64785b6675-sqwxw" (UID: "58e45f2e-a384-481f-b163-fa4accf8c0a1") : secret "prometheus-operator-tls" not found Jul 09 17:41:33.433687 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.433667 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58e45f2e-a384-481f-b163-fa4accf8c0a1-metrics-client-ca\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.435340 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.435316 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58e45f2e-a384-481f-b163-fa4accf8c0a1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.441691 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.441670 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnwd9\" (UniqueName: \"kubernetes.io/projected/58e45f2e-a384-481f-b163-fa4accf8c0a1-kube-api-access-nnwd9\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.937611 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.937580 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/58e45f2e-a384-481f-b163-fa4accf8c0a1-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:33.939905 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:33.939882 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/58e45f2e-a384-481f-b163-fa4accf8c0a1-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-sqwxw\" (UID: \"58e45f2e-a384-481f-b163-fa4accf8c0a1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:34.185227 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:34.185193 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" Jul 09 17:41:34.330701 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:34.328734 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-sqwxw"] Jul 09 17:41:34.331879 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:34.331851 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58e45f2e_a384_481f_b163_fa4accf8c0a1.slice/crio-a0c7585b92650ac57caa77750a94ebf1c461dd0f3b22b85d2a044c846b31e928 WatchSource:0}: Error finding container a0c7585b92650ac57caa77750a94ebf1c461dd0f3b22b85d2a044c846b31e928: Status 404 returned error can't find the container with id a0c7585b92650ac57caa77750a94ebf1c461dd0f3b22b85d2a044c846b31e928 Jul 09 17:41:34.925926 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:34.925882 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" event={"ID":"58e45f2e-a384-481f-b163-fa4accf8c0a1","Type":"ContainerStarted","Data":"a0c7585b92650ac57caa77750a94ebf1c461dd0f3b22b85d2a044c846b31e928"} Jul 09 17:41:35.930778 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:35.930742 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" event={"ID":"58e45f2e-a384-481f-b163-fa4accf8c0a1","Type":"ContainerStarted","Data":"59c4722da67f3f900d1bd4b83834ff7ba7d21f2fb8207d891d2be4ace9707687"} Jul 09 17:41:35.931164 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:35.930798 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" event={"ID":"58e45f2e-a384-481f-b163-fa4accf8c0a1","Type":"ContainerStarted","Data":"16f6db926abbbddc3b8824801975e355f13e2c4bf82ef584bf518ca5f7575a6b"} Jul 09 17:41:35.948181 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:35.948134 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-sqwxw" podStartSLOduration=1.750171351 podStartE2EDuration="2.948120168s" totalImagesPullingTime="1.197948817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:35.947296897 +0000 UTC m=+112.033420386" watchObservedRunningTime="2026-07-09 17:41:35.948120168 +0000 UTC m=+112.034243625" Jul 09 17:41:37.745668 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.745638 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xplkk"] Jul 09 17:41:37.748059 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.748038 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-n7kl8"] Jul 09 17:41:37.748189 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.748171 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.750112 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.750091 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.751682 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.751660 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 17:41:37.751930 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.751914 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 17:41:37.752003 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.751924 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9krtl\"" Jul 09 17:41:37.752061 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.752016 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 17:41:37.752647 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.752630 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 17:41:37.752735 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.752641 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-b6vk4\"" Jul 09 17:41:37.753037 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.753022 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 17:41:37.753089 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.753043 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 17:41:37.766415 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766388 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.766513 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766449 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t986x\" (UniqueName: \"kubernetes.io/projected/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-kube-api-access-t986x\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.766513 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766473 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.766513 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766497 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.766652 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766522 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv897\" (UniqueName: \"kubernetes.io/projected/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-api-access-wv897\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.766652 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766544 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-sys\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.766652 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766575 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-root\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.766652 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766596 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-wtmp\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.766652 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766635 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.766892 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766730 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.766892 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766768 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.766892 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766797 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-textfile\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.766892 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766832 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-tls\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.767073 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766888 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-metrics-client-ca\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.767073 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.766927 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.767420 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.767400 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-n7kl8"] Jul 09 17:41:37.867718 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867680 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.867855 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867727 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t986x\" (UniqueName: \"kubernetes.io/projected/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-kube-api-access-t986x\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.867855 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867747 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.867855 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867775 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.867855 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867801 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv897\" (UniqueName: \"kubernetes.io/projected/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-api-access-wv897\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.867855 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867823 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-sys\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.867855 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867852 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-root\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868135 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867873 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-wtmp\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868135 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867911 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.868135 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867937 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-sys\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868135 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.867944 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.868135 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868002 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868135 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868034 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-textfile\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868135 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868066 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-tls\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868135 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868101 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-metrics-client-ca\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868135 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868135 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.868600 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868137 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868600 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868207 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-root\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868600 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868134 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-wtmp\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868739 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868604 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-textfile\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.868739 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868672 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.868825 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.868791 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-metrics-client-ca\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.869033 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.869006 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.869136 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.869084 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.870896 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.870850 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.871023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.870997 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.871257 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.871237 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-node-exporter-tls\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:37.871480 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.871454 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.876965 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.876941 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv897\" (UniqueName: \"kubernetes.io/projected/7b9fe8bd-8392-422a-a289-f4d2a27dfc40-kube-api-access-wv897\") pod \"kube-state-metrics-785778ff74-n7kl8\" (UID: \"7b9fe8bd-8392-422a-a289-f4d2a27dfc40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:37.880312 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:37.880262 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t986x\" (UniqueName: \"kubernetes.io/projected/9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5-kube-api-access-t986x\") pod \"node-exporter-xplkk\" (UID: \"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5\") " pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:38.061235 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.061160 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xplkk" Jul 09 17:41:38.066918 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.066893 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" Jul 09 17:41:38.072936 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:38.072891 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cfeeab1_ceb4_4922_81dc_3ebf5be9c8f5.slice/crio-b1ab89bd866aa44141aff446beb6d20863376968e6e79cc319d216d21ad59298 WatchSource:0}: Error finding container b1ab89bd866aa44141aff446beb6d20863376968e6e79cc319d216d21ad59298: Status 404 returned error can't find the container with id b1ab89bd866aa44141aff446beb6d20863376968e6e79cc319d216d21ad59298 Jul 09 17:41:38.205246 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.205221 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-n7kl8"] Jul 09 17:41:38.207591 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:38.207559 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b9fe8bd_8392_422a_a289_f4d2a27dfc40.slice/crio-d71994864d9cf96a66043c813e3c7a94387c0cf43ddaa11951c3384561abc869 WatchSource:0}: Error finding container d71994864d9cf96a66043c813e3c7a94387c0cf43ddaa11951c3384561abc869: Status 404 returned error can't find the container with id d71994864d9cf96a66043c813e3c7a94387c0cf43ddaa11951c3384561abc869 Jul 09 17:41:38.720696 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.720656 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:41:38.727757 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.727665 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.730495 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.730319 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 17:41:38.730495 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.730355 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 17:41:38.730495 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.730382 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 17:41:38.730495 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.730319 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 17:41:38.730769 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.730643 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 17:41:38.731030 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.730932 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 17:41:38.731291 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.731205 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 17:41:38.731486 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.731461 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 17:41:38.731681 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.731662 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 17:41:38.731810 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.731784 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xfd85\"" Jul 09 17:41:38.739385 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.739304 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:41:38.775300 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775253 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-out\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775316 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775340 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775400 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775417 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-web-config\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775437 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775461 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775480 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775511 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775533 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775633 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.775666 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775669 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.776065 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.775714 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fxxn\" (UniqueName: \"kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-kube-api-access-5fxxn\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876240 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876198 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876240 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876243 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876509 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876294 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876509 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876321 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876509 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876351 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fxxn\" (UniqueName: \"kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-kube-api-access-5fxxn\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876509 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876395 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-out\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876509 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876423 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876509 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876452 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876509 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876511 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876915 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876533 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-web-config\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876915 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876561 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876915 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876589 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876915 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.876611 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.876915 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:38.876751 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-trusted-ca-bundle podName:1f8f57a9-7636-4ab5-86cd-39fea28b88dc nodeName:}" failed. No retries permitted until 2026-07-09 17:41:39.376730222 +0000 UTC m=+115.462853667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc") : configmap references non-existent config key: ca-bundle.crt Jul 09 17:41:38.878036 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:38.877452 2571 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 09 17:41:38.878036 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.877498 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.878036 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:38.877531 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-main-tls podName:1f8f57a9-7636-4ab5-86cd-39fea28b88dc nodeName:}" failed. No retries permitted until 2026-07-09 17:41:39.377514411 +0000 UTC m=+115.463637876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc") : secret "alertmanager-main-tls" not found Jul 09 17:41:38.878036 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.877570 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.879587 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.879540 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.881318 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.881257 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.882774 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.882733 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.882880 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.882852 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-web-config\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.882953 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.882882 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.883302 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.883282 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.883625 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.883598 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.884573 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.884530 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-out\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.886669 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.886627 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fxxn\" (UniqueName: \"kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-kube-api-access-5fxxn\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:38.901198 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.901166 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7sfwq" Jul 09 17:41:38.945483 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.945376 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" event={"ID":"7b9fe8bd-8392-422a-a289-f4d2a27dfc40","Type":"ContainerStarted","Data":"d71994864d9cf96a66043c813e3c7a94387c0cf43ddaa11951c3384561abc869"} Jul 09 17:41:38.947746 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:38.947667 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xplkk" event={"ID":"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5","Type":"ContainerStarted","Data":"b1ab89bd866aa44141aff446beb6d20863376968e6e79cc319d216d21ad59298"} Jul 09 17:41:39.381600 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.381577 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:39.381694 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.381623 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:39.382383 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.382362 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:39.383711 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.383694 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:39.441025 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.440991 2571 patch_prober.go:28] interesting pod/image-registry-6699c6bb88-4kxjp container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 17:41:39.441162 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.441049 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" podUID="127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 17:41:39.641951 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.641910 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:41:39.939299 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.939260 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:41:39.941460 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:39.941426 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f8f57a9_7636_4ab5_86cd_39fea28b88dc.slice/crio-5688d26eb9e0a2d7fa1d02ea70f2a5ee1ece878fda3122cf06916ac7c99e6a4e WatchSource:0}: Error finding container 5688d26eb9e0a2d7fa1d02ea70f2a5ee1ece878fda3122cf06916ac7c99e6a4e: Status 404 returned error can't find the container with id 5688d26eb9e0a2d7fa1d02ea70f2a5ee1ece878fda3122cf06916ac7c99e6a4e Jul 09 17:41:39.960938 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.960853 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" event={"ID":"7b9fe8bd-8392-422a-a289-f4d2a27dfc40","Type":"ContainerStarted","Data":"39f3f0f26f8a05d91aa152a9aaeb469c80f3ec1ff2183f9af17fdf57d52e2f7e"} Jul 09 17:41:39.962422 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.962389 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerStarted","Data":"5688d26eb9e0a2d7fa1d02ea70f2a5ee1ece878fda3122cf06916ac7c99e6a4e"} Jul 09 17:41:39.963882 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.963858 2571 generic.go:356] "Generic (PLEG): container finished" podID="9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5" containerID="1857ad85ac9d14b62b7fe97e1ffaf5125b7d03a1f2243d37852f982474cb3426" exitCode=0 Jul 09 17:41:39.963958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:39.963890 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xplkk" event={"ID":"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5","Type":"ContainerDied","Data":"1857ad85ac9d14b62b7fe97e1ffaf5125b7d03a1f2243d37852f982474cb3426"} Jul 09 17:41:40.970424 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:40.970380 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xplkk" event={"ID":"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5","Type":"ContainerStarted","Data":"cb177a2486b57fc8767d74dce5de1c7d3cfcec13ef10760b25b9d29c7f7cb36e"} Jul 09 17:41:40.970845 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:40.970458 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xplkk" event={"ID":"9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5","Type":"ContainerStarted","Data":"bc95811f97d20f313e1641fecd4aae6d0eabfaf169131e1ca22576cf4db60f70"} Jul 09 17:41:40.972747 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:40.972721 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" event={"ID":"7b9fe8bd-8392-422a-a289-f4d2a27dfc40","Type":"ContainerStarted","Data":"5f7abe9fd089599de072fd1aecd3560e30f2432fa85a42a368060caadc080ea2"} Jul 09 17:41:40.972865 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:40.972778 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" event={"ID":"7b9fe8bd-8392-422a-a289-f4d2a27dfc40","Type":"ContainerStarted","Data":"5b60cd7cab579a4089734652ffaf494b68f3a7ad11157619267ac2ee1f5ca95d"} Jul 09 17:41:40.993670 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:40.993625 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xplkk" podStartSLOduration=2.774317782 podStartE2EDuration="3.993610831s" totalImagesPullingTime="1.154260376s" totalInitContainerRuntime="65.032673ms" isStatefulPod=true podCreationTimestamp="2026-07-09 17:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:40.991968821 +0000 UTC m=+117.078092291" watchObservedRunningTime="2026-07-09 17:41:40.993610831 +0000 UTC m=+117.079734298" Jul 09 17:41:41.017626 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:41.017580 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-n7kl8" podStartSLOduration=2.371162437 podStartE2EDuration="4.017565895s" totalImagesPullingTime="1.646403458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:41.016596654 +0000 UTC m=+117.102720142" watchObservedRunningTime="2026-07-09 17:41:41.017565895 +0000 UTC m=+117.103689734" Jul 09 17:41:41.977826 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:41.977792 2571 generic.go:356] "Generic (PLEG): container finished" podID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerID="801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4" exitCode=0 Jul 09 17:41:41.978187 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:41.977870 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerDied","Data":"801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4"} Jul 09 17:41:42.393255 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.393220 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx"] Jul 09 17:41:42.396479 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.396459 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:42.398956 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.398936 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 09 17:41:42.399144 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.399124 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cbgfm\"" Jul 09 17:41:42.405925 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.405906 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx"] Jul 09 17:41:42.510071 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.510042 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt9hp\" (UniqueName: \"kubernetes.io/projected/0a4844e3-3459-4990-838c-b403938b04a3-kube-api-access-xt9hp\") pod \"monitoring-plugin-5f68df7f58-zvsfx\" (UID: \"0a4844e3-3459-4990-838c-b403938b04a3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:42.510227 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.510170 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0a4844e3-3459-4990-838c-b403938b04a3-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-zvsfx\" (UID: \"0a4844e3-3459-4990-838c-b403938b04a3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:42.611408 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.611374 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0a4844e3-3459-4990-838c-b403938b04a3-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-zvsfx\" (UID: \"0a4844e3-3459-4990-838c-b403938b04a3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:42.611540 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.611416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt9hp\" (UniqueName: \"kubernetes.io/projected/0a4844e3-3459-4990-838c-b403938b04a3-kube-api-access-xt9hp\") pod \"monitoring-plugin-5f68df7f58-zvsfx\" (UID: \"0a4844e3-3459-4990-838c-b403938b04a3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:42.611540 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:42.611525 2571 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 09 17:41:42.611618 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:42.611594 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a4844e3-3459-4990-838c-b403938b04a3-monitoring-plugin-cert podName:0a4844e3-3459-4990-838c-b403938b04a3 nodeName:}" failed. No retries permitted until 2026-07-09 17:41:43.111572476 +0000 UTC m=+119.197695920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/0a4844e3-3459-4990-838c-b403938b04a3-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-zvsfx" (UID: "0a4844e3-3459-4990-838c-b403938b04a3") : secret "monitoring-plugin-cert" not found Jul 09 17:41:42.615479 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.615448 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84465d6d7d-5s9jh"] Jul 09 17:41:42.619604 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.619576 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.622793 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.622524 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pqnxv\"" Jul 09 17:41:42.622793 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.622543 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 17:41:42.622793 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.622528 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 17:41:42.622793 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.622543 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 17:41:42.622793 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.622545 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 17:41:42.622793 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.622651 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 17:41:42.623139 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.622919 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 17:41:42.623139 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.622977 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 17:41:42.627740 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.627708 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt9hp\" (UniqueName: \"kubernetes.io/projected/0a4844e3-3459-4990-838c-b403938b04a3-kube-api-access-xt9hp\") pod \"monitoring-plugin-5f68df7f58-zvsfx\" (UID: \"0a4844e3-3459-4990-838c-b403938b04a3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:42.628517 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.628494 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84465d6d7d-5s9jh"] Jul 09 17:41:42.712008 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.711926 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-serving-cert\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.712008 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.711964 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jh8j\" (UniqueName: \"kubernetes.io/projected/5422e158-97d6-4d3a-a189-a790daa14a6a-kube-api-access-5jh8j\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.712213 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.712027 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-console-config\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.712213 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.712068 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-oauth-serving-cert\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.712213 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.712148 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-oauth-config\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.712213 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.712193 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-service-ca\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.813077 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.813027 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-serving-cert\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.813077 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.813068 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jh8j\" (UniqueName: \"kubernetes.io/projected/5422e158-97d6-4d3a-a189-a790daa14a6a-kube-api-access-5jh8j\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.813356 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.813107 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-console-config\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.813356 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.813158 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-oauth-serving-cert\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.813356 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.813193 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-oauth-config\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.813356 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.813218 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-service-ca\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.813937 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.813913 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-console-config\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.813937 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.813923 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-oauth-serving-cert\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.814098 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.814063 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-service-ca\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.815576 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.815548 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-oauth-config\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.815710 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.815690 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-serving-cert\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.828215 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.828168 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jh8j\" (UniqueName: \"kubernetes.io/projected/5422e158-97d6-4d3a-a189-a790daa14a6a-kube-api-access-5jh8j\") pod \"console-84465d6d7d-5s9jh\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:42.898214 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.898183 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5f759875c-np9h5"] Jul 09 17:41:42.903036 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.903016 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:42.906236 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.906213 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 09 17:41:42.906364 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.906305 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 09 17:41:42.906606 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.906521 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 09 17:41:42.906606 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.906521 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 09 17:41:42.906922 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.906903 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jl4rj\"" Jul 09 17:41:42.908764 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.908744 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 09 17:41:42.911351 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.911328 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 09 17:41:42.918473 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.918453 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f759875c-np9h5"] Jul 09 17:41:42.939730 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:42.939708 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:43.014989 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.014657 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-serving-certs-ca-bundle\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.014989 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.014707 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.014989 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.014751 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-metrics-client-ca\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.014989 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.014793 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-federate-client-tls\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.014989 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.014835 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.014989 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.014867 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj29l\" (UniqueName: \"kubernetes.io/projected/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-kube-api-access-kj29l\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.014989 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.014899 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-secret-telemeter-client\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.014989 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.014934 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-telemeter-client-tls\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.095550 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.095471 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84465d6d7d-5s9jh"] Jul 09 17:41:43.099620 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:43.099583 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5422e158_97d6_4d3a_a189_a790daa14a6a.slice/crio-327f0e98fcebb2282d325ca745d8bf49ab8709c484bcc9c783e3e3b53676fd55 WatchSource:0}: Error finding container 327f0e98fcebb2282d325ca745d8bf49ab8709c484bcc9c783e3e3b53676fd55: Status 404 returned error can't find the container with id 327f0e98fcebb2282d325ca745d8bf49ab8709c484bcc9c783e3e3b53676fd55 Jul 09 17:41:43.115708 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.115681 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.115821 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.115720 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj29l\" (UniqueName: \"kubernetes.io/projected/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-kube-api-access-kj29l\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.115899 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.115856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-secret-telemeter-client\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.115959 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.115916 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-telemeter-client-tls\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.115959 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.115953 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0a4844e3-3459-4990-838c-b403938b04a3-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-zvsfx\" (UID: \"0a4844e3-3459-4990-838c-b403938b04a3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:43.116082 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.116065 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-serving-certs-ca-bundle\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.116144 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.116097 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.116144 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.116136 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-metrics-client-ca\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.116248 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.116175 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-federate-client-tls\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.116784 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.116732 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-serving-certs-ca-bundle\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.116784 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.116750 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.117001 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.116954 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-metrics-client-ca\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.118869 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.118845 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-secret-telemeter-client\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.118969 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.118873 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0a4844e3-3459-4990-838c-b403938b04a3-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-zvsfx\" (UID: \"0a4844e3-3459-4990-838c-b403938b04a3\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:43.119028 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.118961 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.119028 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.118990 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-telemeter-client-tls\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.119174 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.119151 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-federate-client-tls\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.126066 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.126045 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj29l\" (UniqueName: \"kubernetes.io/projected/f1b32223-ef89-4e8a-84eb-0c9b51ea2213-kube-api-access-kj29l\") pod \"telemeter-client-5f759875c-np9h5\" (UID: \"f1b32223-ef89-4e8a-84eb-0c9b51ea2213\") " pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.214836 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.214799 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" Jul 09 17:41:43.305746 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.305476 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:43.364020 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.363716 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f759875c-np9h5"] Jul 09 17:41:43.366835 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:43.366789 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b32223_ef89_4e8a_84eb_0c9b51ea2213.slice/crio-67bdf99eea1258e7860116a911bb7715b914ee09e280714d4f2a30a1c187f58e WatchSource:0}: Error finding container 67bdf99eea1258e7860116a911bb7715b914ee09e280714d4f2a30a1c187f58e: Status 404 returned error can't find the container with id 67bdf99eea1258e7860116a911bb7715b914ee09e280714d4f2a30a1c187f58e Jul 09 17:41:43.442770 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.442738 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx"] Jul 09 17:41:43.745455 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:43.745425 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4844e3_3459_4990_838c_b403938b04a3.slice/crio-4be1dc43daa8272fb575080ab726f8af5239ede8c01b4d27323903c49e2858e9 WatchSource:0}: Error finding container 4be1dc43daa8272fb575080ab726f8af5239ede8c01b4d27323903c49e2858e9: Status 404 returned error can't find the container with id 4be1dc43daa8272fb575080ab726f8af5239ede8c01b4d27323903c49e2858e9 Jul 09 17:41:43.949138 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.948611 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:41:43.954819 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.954796 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:43.958295 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.958005 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 17:41:43.960035 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.959713 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 17:41:43.960035 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.959978 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 17:41:43.960193 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.960182 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 17:41:43.960361 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.960326 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 17:41:43.960458 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.960417 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 17:41:43.960524 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.960507 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 17:41:43.960931 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.960901 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 17:41:43.962702 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.961055 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-76aqvjiju0qrf\"" Jul 09 17:41:43.962702 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.960926 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-77p6b\"" Jul 09 17:41:43.962702 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.960982 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 17:41:43.962702 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.961350 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 17:41:43.962702 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.961526 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 17:41:43.962702 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.961778 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 17:41:43.962702 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.962038 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 17:41:43.968758 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.966457 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 17:41:43.969305 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.969062 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:41:43.970413 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.969933 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 17:41:43.994627 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.994502 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" event={"ID":"0a4844e3-3459-4990-838c-b403938b04a3","Type":"ContainerStarted","Data":"4be1dc43daa8272fb575080ab726f8af5239ede8c01b4d27323903c49e2858e9"} Jul 09 17:41:43.997239 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.997172 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" event={"ID":"f1b32223-ef89-4e8a-84eb-0c9b51ea2213","Type":"ContainerStarted","Data":"67bdf99eea1258e7860116a911bb7715b914ee09e280714d4f2a30a1c187f58e"} Jul 09 17:41:43.998470 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:43.998442 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84465d6d7d-5s9jh" event={"ID":"5422e158-97d6-4d3a-a189-a790daa14a6a","Type":"ContainerStarted","Data":"327f0e98fcebb2282d325ca745d8bf49ab8709c484bcc9c783e3e3b53676fd55"} Jul 09 17:41:44.004121 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.004074 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerStarted","Data":"d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03"} Jul 09 17:41:44.004121 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.004179 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerStarted","Data":"0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf"} Jul 09 17:41:44.025972 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.025932 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.026547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.025993 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.026547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026023 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.026547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026055 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.026547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026080 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.026547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026147 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tktzm\" (UniqueName: \"kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-kube-api-access-tktzm\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.026547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026229 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.026547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026336 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.026547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026402 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.026547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026501 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.026547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026530 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.027026 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026582 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.027026 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026616 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.027026 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026637 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.027026 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026679 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.027026 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026706 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.027026 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026744 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.027026 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026759 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.027026 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026807 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.027026 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.026853 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.127902 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.127949 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.127974 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128019 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128048 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128071 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128101 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128127 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128156 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128181 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128209 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tktzm\" (UniqueName: \"kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-kube-api-access-tktzm\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128240 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128292 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128333 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128397 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.128419 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.129304 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.129253 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.134646 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.129318 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.134646 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.129367 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.134646 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.129400 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.134646 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.129430 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.134646 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.129927 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.134646 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.130524 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.134646 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.130960 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.136867 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.135703 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.136867 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.136557 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.137812 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.137786 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.137915 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.137888 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.140023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.139142 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.140023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.139597 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.140023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.139853 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.140023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.139977 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.141384 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.141334 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.141490 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.141466 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.141940 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.141913 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.142879 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.142832 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.143448 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.143397 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.143700 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.143656 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.145448 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.145394 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.145559 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.145528 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tktzm\" (UniqueName: \"kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-kube-api-access-tktzm\") pod \"prometheus-k8s-0\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.295044 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.294972 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-77p6b\"" Jul 09 17:41:44.303445 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.303418 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:44.505185 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:44.505102 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:41:44.510681 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:44.510652 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4059f1d_a41c_4a56_960b_03984ad6a0e4.slice/crio-d1d7be2278cfda8131eccaf95375ac9b660aa575544cdee07536fab84182cf3b WatchSource:0}: Error finding container d1d7be2278cfda8131eccaf95375ac9b660aa575544cdee07536fab84182cf3b: Status 404 returned error can't find the container with id d1d7be2278cfda8131eccaf95375ac9b660aa575544cdee07536fab84182cf3b Jul 09 17:41:45.016799 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:45.016561 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerStarted","Data":"7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf"} Jul 09 17:41:45.016799 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:45.016699 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerStarted","Data":"7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e"} Jul 09 17:41:45.016799 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:45.016748 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerStarted","Data":"1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd"} Jul 09 17:41:45.018951 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:45.018922 2571 generic.go:356] "Generic (PLEG): container finished" podID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerID="17a88c13c2d72451dd5bdb9874b73c841ee578baa95f49391d7b3eaeee33cca9" exitCode=0 Jul 09 17:41:45.019059 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:45.018973 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerDied","Data":"17a88c13c2d72451dd5bdb9874b73c841ee578baa95f49391d7b3eaeee33cca9"} Jul 09 17:41:45.019059 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:45.019032 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerStarted","Data":"d1d7be2278cfda8131eccaf95375ac9b660aa575544cdee07536fab84182cf3b"} Jul 09 17:41:48.035156 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.035047 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84465d6d7d-5s9jh" event={"ID":"5422e158-97d6-4d3a-a189-a790daa14a6a","Type":"ContainerStarted","Data":"cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e"} Jul 09 17:41:48.039157 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.039119 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerStarted","Data":"79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2"} Jul 09 17:41:48.040872 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.040847 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" event={"ID":"0a4844e3-3459-4990-838c-b403938b04a3","Type":"ContainerStarted","Data":"9c5675e949a6e66bf77c63bf131ee45eb9f17ce35863b62a7520b7dfffa5c8e8"} Jul 09 17:41:48.041064 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.041045 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:48.043146 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.043121 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" event={"ID":"f1b32223-ef89-4e8a-84eb-0c9b51ea2213","Type":"ContainerStarted","Data":"cb4d4df5288609d6e2da6039f68a20e270febcbd8567bfe48a2730c350d626e6"} Jul 09 17:41:48.043244 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.043172 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" event={"ID":"f1b32223-ef89-4e8a-84eb-0c9b51ea2213","Type":"ContainerStarted","Data":"5307b9de65ed29bfd67c25507340eac763194e1107bab0cf4ea2e41919522a83"} Jul 09 17:41:48.043244 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.043201 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" event={"ID":"f1b32223-ef89-4e8a-84eb-0c9b51ea2213","Type":"ContainerStarted","Data":"d3fa8528cdd97bb248cea98d545f81c73f349da01ab12467531b17b055194c7d"} Jul 09 17:41:48.046801 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.046782 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" Jul 09 17:41:48.057685 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.057636 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84465d6d7d-5s9jh" podStartSLOduration=2.057039988 podStartE2EDuration="6.057617904s" totalImagesPullingTime="4.000577916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:48.055402655 +0000 UTC m=+124.141526141" watchObservedRunningTime="2026-07-09 17:41:48.057617904 +0000 UTC m=+124.143741372" Jul 09 17:41:48.081545 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.081496 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.276647028 podStartE2EDuration="10.081482541s" totalImagesPullingTime="5.788788338s" totalInitContainerRuntime="16.047175ms" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:48.08021226 +0000 UTC m=+124.166335738" watchObservedRunningTime="2026-07-09 17:41:48.081482541 +0000 UTC m=+124.167606005" Jul 09 17:41:48.096873 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:48.096822 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zvsfx" podStartSLOduration=2.7976827589999997 podStartE2EDuration="6.096803475s" totalImagesPullingTime="3.299120716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:48.096329057 +0000 UTC m=+124.182452539" watchObservedRunningTime="2026-07-09 17:41:48.096803475 +0000 UTC m=+124.182926942" Jul 09 17:41:49.447212 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:49.447183 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:49.477556 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:49.477491 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5f759875c-np9h5" podStartSLOduration=3.796731364 podStartE2EDuration="7.477467298s" totalImagesPullingTime="3.680735934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:48.118122586 +0000 UTC m=+124.204246079" watchObservedRunningTime="2026-07-09 17:41:49.477467298 +0000 UTC m=+125.563590763" Jul 09 17:41:50.065493 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:50.065452 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerStarted","Data":"f5a3c2a4e502328162b39daf667eecea7e855132568eba15cddcfbe77f3f2b84"} Jul 09 17:41:50.065680 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:50.065577 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerStarted","Data":"8c18239f1b2f7e74166f468734093fe9185ed74d24c307caec207e633d225a16"} Jul 09 17:41:52.076022 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:52.075983 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" event={"ID":"4e0be933-6aac-4a1b-964e-f43b1ec01455","Type":"ContainerStarted","Data":"53a334ef4114a6ab4909801f0d6dcac81ca3c72cf900c7828f79877ad4cb810f"} Jul 09 17:41:52.079325 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:52.079297 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerStarted","Data":"8d83ee3562d9d2eba7adbff19d948af25b2078f9672c4ba1d4a78af791618e09"} Jul 09 17:41:52.079452 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:52.079371 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerStarted","Data":"1e114bc12bfb4492fe41a627b9a6cee856be114368e79b2c10936f507e057ed4"} Jul 09 17:41:52.079452 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:52.079419 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerStarted","Data":"d4c4e2d6a9b704e13c3a65ecb0f629a9d121b2f9db65523b712b20516b6b30db"} Jul 09 17:41:52.079558 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:52.079457 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerStarted","Data":"0d28336cfda212693442a8f074796070e5dbbf5b8151acbdf5ebba07c502c162"} Jul 09 17:41:52.097091 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:52.097048 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f5f69fdc4-2tgvz" podStartSLOduration=1.416732394 podStartE2EDuration="23.097032937s" totalImagesPullingTime="21.680300543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:52.096127003 +0000 UTC m=+128.182250470" watchObservedRunningTime="2026-07-09 17:41:52.097032937 +0000 UTC m=+128.183156406" Jul 09 17:41:52.132981 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:52.132929 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.755951358 podStartE2EDuration="9.132908406s" totalImagesPullingTime="6.348571748s" totalInitContainerRuntime="28.3853ms" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:52.131509933 +0000 UTC m=+128.217633459" watchObservedRunningTime="2026-07-09 17:41:52.132908406 +0000 UTC m=+128.219031874" Jul 09 17:41:52.939919 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:52.939887 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:52.940086 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:52.939975 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:52.944679 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:52.944655 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:53.087373 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:53.087346 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:41:54.241150 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.241114 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:41:54.243437 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.243418 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02523ead-082e-41e1-ab84-648a74b97bd2-metrics-certs\") pod \"network-metrics-daemon-lwssb\" (UID: \"02523ead-082e-41e1-ab84-648a74b97bd2\") " pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:41:54.294213 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.294183 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f99f5\"" Jul 09 17:41:54.302315 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.302295 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwssb" Jul 09 17:41:54.304538 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.304520 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:41:54.448442 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.448413 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lwssb"] Jul 09 17:41:54.451919 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:41:54.451895 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02523ead_082e_41e1_ab84_648a74b97bd2.slice/crio-c6925cf732a8410b5bf0f84f637d4d6fdd3d0c9ce29db4347a2ab8e8e6ff524d WatchSource:0}: Error finding container c6925cf732a8410b5bf0f84f637d4d6fdd3d0c9ce29db4347a2ab8e8e6ff524d: Status 404 returned error can't find the container with id c6925cf732a8410b5bf0f84f637d4d6fdd3d0c9ce29db4347a2ab8e8e6ff524d Jul 09 17:41:54.455430 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.455375 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" podUID="127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" containerName="registry" containerID="cri-o://b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1" gracePeriod=30 Jul 09 17:41:54.708843 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.708820 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:54.846816 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.846792 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-installation-pull-secrets\") pod \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " Jul 09 17:41:54.846981 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.846833 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-image-registry-private-configuration\") pod \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " Jul 09 17:41:54.846981 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.846900 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls\") pod \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " Jul 09 17:41:54.846981 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.846934 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-certificates\") pod \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " Jul 09 17:41:54.846981 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.846961 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-ca-trust-extracted\") pod \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " Jul 09 17:41:54.847176 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.847088 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-trusted-ca\") pod \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " Jul 09 17:41:54.847176 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.847140 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-bound-sa-token\") pod \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " Jul 09 17:41:54.847300 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.847204 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mw8n\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-kube-api-access-7mw8n\") pod \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\" (UID: \"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a\") " Jul 09 17:41:54.847507 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.847476 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:41:54.847507 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.847476 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:41:54.849253 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.849225 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:41:54.849404 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.849251 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-kube-api-access-7mw8n" (OuterVolumeSpecName: "kube-api-access-7mw8n") pod "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a"). InnerVolumeSpecName "kube-api-access-7mw8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:41:54.849485 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.849464 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:41:54.849540 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.849476 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:41:54.849540 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.849524 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:41:54.855742 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.855719 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" (UID: "127cf5c0-21b8-4fb4-a147-a12d89d6ef7a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:54.948687 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.948658 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mw8n\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-kube-api-access-7mw8n\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:41:54.948687 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.948684 2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-installation-pull-secrets\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:41:54.948850 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.948696 2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-image-registry-private-configuration\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:41:54.948850 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.948707 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-tls\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:41:54.948850 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.948716 2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-registry-certificates\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:41:54.948850 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.948724 2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-ca-trust-extracted\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:41:54.948850 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.948732 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-trusted-ca\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:41:54.948850 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:54.948740 2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a-bound-sa-token\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:41:55.093979 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:55.093934 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lwssb" event={"ID":"02523ead-082e-41e1-ab84-648a74b97bd2","Type":"ContainerStarted","Data":"c6925cf732a8410b5bf0f84f637d4d6fdd3d0c9ce29db4347a2ab8e8e6ff524d"} Jul 09 17:41:55.095794 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:55.095758 2571 generic.go:356] "Generic (PLEG): container finished" podID="127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" containerID="b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1" exitCode=0 Jul 09 17:41:55.095926 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:55.095817 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" Jul 09 17:41:55.095926 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:55.095850 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" event={"ID":"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a","Type":"ContainerDied","Data":"b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1"} Jul 09 17:41:55.095926 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:55.095904 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6699c6bb88-4kxjp" event={"ID":"127cf5c0-21b8-4fb4-a147-a12d89d6ef7a","Type":"ContainerDied","Data":"3e8863c1098a273f426ba8029a13795997660f9f29b972c58d659897b9c32a5b"} Jul 09 17:41:55.095926 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:55.095927 2571 scope.go:117] "RemoveContainer" containerID="b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1" Jul 09 17:41:55.111384 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:55.111358 2571 scope.go:117] "RemoveContainer" containerID="b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1" Jul 09 17:41:55.111786 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:41:55.111758 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1\": container with ID starting with b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1 not found: ID does not exist" containerID="b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1" Jul 09 17:41:55.111869 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:55.111797 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1"} err="failed to get container status \"b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1\": rpc error: code = NotFound desc = could not find container \"b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1\": container with ID starting with b3b4b5611cb147a6e2416c9748ede5753f304806c6806c28c962b46e326d6cd1 not found: ID does not exist" Jul 09 17:41:55.124926 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:55.124886 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6699c6bb88-4kxjp"] Jul 09 17:41:55.131560 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:55.131536 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6699c6bb88-4kxjp"] Jul 09 17:41:56.102251 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:56.102215 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lwssb" event={"ID":"02523ead-082e-41e1-ab84-648a74b97bd2","Type":"ContainerStarted","Data":"4a3a448f77bae3a6c3d52518b6bc891e9291623fd8d0c4d7c2b1a42b0af04c9c"} Jul 09 17:41:56.102683 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:56.102334 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lwssb" event={"ID":"02523ead-082e-41e1-ab84-648a74b97bd2","Type":"ContainerStarted","Data":"99e191f79e1a1bd1adf84779e37f2e1e921be3a44299f4360266ac7ca82d7b16"} Jul 09 17:41:56.120796 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:56.120743 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lwssb" podStartSLOduration=130.118439656 podStartE2EDuration="2m11.12072557s" totalImagesPullingTime="1.002285914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:56.119503349 +0000 UTC m=+132.205626815" watchObservedRunningTime="2026-07-09 17:41:56.12072557 +0000 UTC m=+132.206849036" Jul 09 17:41:56.477974 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:41:56.477896 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" path="/var/lib/kubelet/pods/127cf5c0-21b8-4fb4-a147-a12d89d6ef7a/volumes" Jul 09 17:42:04.410111 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:04.410077 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84465d6d7d-5s9jh"] Jul 09 17:42:29.431481 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.431441 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84465d6d7d-5s9jh" podUID="5422e158-97d6-4d3a-a189-a790daa14a6a" containerName="console" containerID="cri-o://cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e" gracePeriod=15 Jul 09 17:42:29.675935 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.675910 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:42:29.721718 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.721639 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-console-config\") pod \"5422e158-97d6-4d3a-a189-a790daa14a6a\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " Jul 09 17:42:29.721718 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.721692 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-oauth-serving-cert\") pod \"5422e158-97d6-4d3a-a189-a790daa14a6a\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " Jul 09 17:42:29.721922 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.721750 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-serving-cert\") pod \"5422e158-97d6-4d3a-a189-a790daa14a6a\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " Jul 09 17:42:29.721922 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.721777 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-oauth-config\") pod \"5422e158-97d6-4d3a-a189-a790daa14a6a\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " Jul 09 17:42:29.721922 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.721807 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-service-ca\") pod \"5422e158-97d6-4d3a-a189-a790daa14a6a\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " Jul 09 17:42:29.721922 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.721845 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jh8j\" (UniqueName: \"kubernetes.io/projected/5422e158-97d6-4d3a-a189-a790daa14a6a-kube-api-access-5jh8j\") pod \"5422e158-97d6-4d3a-a189-a790daa14a6a\" (UID: \"5422e158-97d6-4d3a-a189-a790daa14a6a\") " Jul 09 17:42:29.722180 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.722152 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-console-config" (OuterVolumeSpecName: "console-config") pod "5422e158-97d6-4d3a-a189-a790daa14a6a" (UID: "5422e158-97d6-4d3a-a189-a790daa14a6a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:42:29.722237 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.722161 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5422e158-97d6-4d3a-a189-a790daa14a6a" (UID: "5422e158-97d6-4d3a-a189-a790daa14a6a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:42:29.722312 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.722261 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-service-ca" (OuterVolumeSpecName: "service-ca") pod "5422e158-97d6-4d3a-a189-a790daa14a6a" (UID: "5422e158-97d6-4d3a-a189-a790daa14a6a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:42:29.724110 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.724086 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5422e158-97d6-4d3a-a189-a790daa14a6a" (UID: "5422e158-97d6-4d3a-a189-a790daa14a6a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:42:29.724241 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.724192 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5422e158-97d6-4d3a-a189-a790daa14a6a-kube-api-access-5jh8j" (OuterVolumeSpecName: "kube-api-access-5jh8j") pod "5422e158-97d6-4d3a-a189-a790daa14a6a" (UID: "5422e158-97d6-4d3a-a189-a790daa14a6a"). InnerVolumeSpecName "kube-api-access-5jh8j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:42:29.724384 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.724359 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5422e158-97d6-4d3a-a189-a790daa14a6a" (UID: "5422e158-97d6-4d3a-a189-a790daa14a6a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:42:29.823488 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.823458 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-oauth-serving-cert\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:29.823488 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.823484 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-serving-cert\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:29.823488 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.823494 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5422e158-97d6-4d3a-a189-a790daa14a6a-console-oauth-config\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:29.823734 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.823504 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-service-ca\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:29.823734 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.823513 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jh8j\" (UniqueName: \"kubernetes.io/projected/5422e158-97d6-4d3a-a189-a790daa14a6a-kube-api-access-5jh8j\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:29.823734 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:29.823520 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5422e158-97d6-4d3a-a189-a790daa14a6a-console-config\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:30.241708 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:30.241669 2571 generic.go:356] "Generic (PLEG): container finished" podID="5422e158-97d6-4d3a-a189-a790daa14a6a" containerID="cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e" exitCode=0 Jul 09 17:42:30.241896 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:30.241732 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84465d6d7d-5s9jh" Jul 09 17:42:30.241896 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:30.241749 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84465d6d7d-5s9jh" event={"ID":"5422e158-97d6-4d3a-a189-a790daa14a6a","Type":"ContainerDied","Data":"cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e"} Jul 09 17:42:30.241896 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:30.241832 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84465d6d7d-5s9jh" event={"ID":"5422e158-97d6-4d3a-a189-a790daa14a6a","Type":"ContainerDied","Data":"327f0e98fcebb2282d325ca745d8bf49ab8709c484bcc9c783e3e3b53676fd55"} Jul 09 17:42:30.241896 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:30.241859 2571 scope.go:117] "RemoveContainer" containerID="cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e" Jul 09 17:42:30.251935 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:30.251921 2571 scope.go:117] "RemoveContainer" containerID="cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e" Jul 09 17:42:30.252201 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:42:30.252183 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e\": container with ID starting with cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e not found: ID does not exist" containerID="cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e" Jul 09 17:42:30.252243 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:30.252208 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e"} err="failed to get container status \"cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e\": rpc error: code = NotFound desc = could not find container \"cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e\": container with ID starting with cf2e8531b6a317e1a14a99022ac316b7f598db5d4a1d00efe81065135e00ac2e not found: ID does not exist" Jul 09 17:42:30.263707 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:30.263676 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84465d6d7d-5s9jh"] Jul 09 17:42:30.267209 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:30.267182 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84465d6d7d-5s9jh"] Jul 09 17:42:30.478056 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:30.478021 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5422e158-97d6-4d3a-a189-a790daa14a6a" path="/var/lib/kubelet/pods/5422e158-97d6-4d3a-a189-a790daa14a6a/volumes" Jul 09 17:42:44.303886 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:44.303861 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:42:44.322942 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:44.322921 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:42:45.318217 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:45.318190 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:42:58.015100 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.015059 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:42:58.015742 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.015684 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="alertmanager" containerID="cri-o://0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf" gracePeriod=120 Jul 09 17:42:58.019302 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.016114 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy" containerID="cri-o://7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e" gracePeriod=120 Jul 09 17:42:58.019302 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.016306 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="prom-label-proxy" containerID="cri-o://79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2" gracePeriod=120 Jul 09 17:42:58.019302 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.016372 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy-metric" containerID="cri-o://7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf" gracePeriod=120 Jul 09 17:42:58.019302 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.016437 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="config-reloader" containerID="cri-o://d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03" gracePeriod=120 Jul 09 17:42:58.019302 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.016532 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy-web" containerID="cri-o://1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd" gracePeriod=120 Jul 09 17:42:58.355315 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.355261 2571 generic.go:356] "Generic (PLEG): container finished" podID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerID="79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2" exitCode=0 Jul 09 17:42:58.355315 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.355300 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerDied","Data":"79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2"} Jul 09 17:42:58.355497 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.355366 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerDied","Data":"7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e"} Jul 09 17:42:58.355497 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.355308 2571 generic.go:356] "Generic (PLEG): container finished" podID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerID="7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e" exitCode=0 Jul 09 17:42:58.355497 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.355399 2571 generic.go:356] "Generic (PLEG): container finished" podID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerID="d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03" exitCode=0 Jul 09 17:42:58.355497 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.355403 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerDied","Data":"d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03"} Jul 09 17:42:58.355497 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.355409 2571 generic.go:356] "Generic (PLEG): container finished" podID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerID="0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf" exitCode=0 Jul 09 17:42:58.355497 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:58.355441 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerDied","Data":"0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf"} Jul 09 17:42:59.263747 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.263723 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.362683 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.362654 2571 generic.go:356] "Generic (PLEG): container finished" podID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerID="7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf" exitCode=0 Jul 09 17:42:59.362683 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.362678 2571 generic.go:356] "Generic (PLEG): container finished" podID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerID="1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd" exitCode=0 Jul 09 17:42:59.362871 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.362720 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerDied","Data":"7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf"} Jul 09 17:42:59.362871 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.362741 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.362871 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.362777 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerDied","Data":"1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd"} Jul 09 17:42:59.362871 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.362804 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f8f57a9-7636-4ab5-86cd-39fea28b88dc","Type":"ContainerDied","Data":"5688d26eb9e0a2d7fa1d02ea70f2a5ee1ece878fda3122cf06916ac7c99e6a4e"} Jul 09 17:42:59.362871 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.362833 2571 scope.go:117] "RemoveContainer" containerID="79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2" Jul 09 17:42:59.370045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.370029 2571 scope.go:117] "RemoveContainer" containerID="7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf" Jul 09 17:42:59.376632 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.376619 2571 scope.go:117] "RemoveContainer" containerID="7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e" Jul 09 17:42:59.383007 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.382989 2571 scope.go:117] "RemoveContainer" containerID="1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd" Jul 09 17:42:59.389066 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389043 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-web-config\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389167 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389071 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-volume\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389167 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389103 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389167 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389132 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-web\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389384 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389232 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fxxn\" (UniqueName: \"kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-kube-api-access-5fxxn\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389384 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389332 2571 scope.go:117] "RemoveContainer" containerID="d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03" Jul 09 17:42:59.389547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389409 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-main-tls\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389475 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-trusted-ca-bundle\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389547 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389539 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389703 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389570 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-metrics-client-ca\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389703 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389612 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-main-db\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389703 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389638 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-tls-assets\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389703 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389664 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-out\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.389703 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.389686 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-cluster-tls-config\") pod \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\" (UID: \"1f8f57a9-7636-4ab5-86cd-39fea28b88dc\") " Jul 09 17:42:59.390782 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.390069 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:42:59.390782 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.390489 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:42:59.391639 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.391569 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:59.392799 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.392609 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:42:59.392799 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.392706 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-kube-api-access-5fxxn" (OuterVolumeSpecName: "kube-api-access-5fxxn") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "kube-api-access-5fxxn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:42:59.392799 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.392786 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-volume" (OuterVolumeSpecName: "config-volume") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:42:59.393469 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.393431 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:42:59.393659 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.393599 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:42:59.393659 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.393625 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:42:59.393786 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.393719 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:42:59.393881 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.393863 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-out" (OuterVolumeSpecName: "config-out") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:59.397144 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.397117 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:42:59.403559 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.403539 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-web-config" (OuterVolumeSpecName: "web-config") pod "1f8f57a9-7636-4ab5-86cd-39fea28b88dc" (UID: "1f8f57a9-7636-4ab5-86cd-39fea28b88dc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:42:59.405640 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.405627 2571 scope.go:117] "RemoveContainer" containerID="0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf" Jul 09 17:42:59.412436 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.412416 2571 scope.go:117] "RemoveContainer" containerID="801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4" Jul 09 17:42:59.419018 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.419003 2571 scope.go:117] "RemoveContainer" containerID="79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2" Jul 09 17:42:59.419244 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:42:59.419225 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2\": container with ID starting with 79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2 not found: ID does not exist" containerID="79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2" Jul 09 17:42:59.419381 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.419251 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2"} err="failed to get container status \"79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2\": rpc error: code = NotFound desc = could not find container \"79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2\": container with ID starting with 79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2 not found: ID does not exist" Jul 09 17:42:59.419381 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.419291 2571 scope.go:117] "RemoveContainer" containerID="7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf" Jul 09 17:42:59.419533 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:42:59.419516 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf\": container with ID starting with 7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf not found: ID does not exist" containerID="7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf" Jul 09 17:42:59.419570 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.419539 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf"} err="failed to get container status \"7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf\": rpc error: code = NotFound desc = could not find container \"7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf\": container with ID starting with 7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf not found: ID does not exist" Jul 09 17:42:59.419570 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.419553 2571 scope.go:117] "RemoveContainer" containerID="7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e" Jul 09 17:42:59.419800 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:42:59.419782 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e\": container with ID starting with 7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e not found: ID does not exist" containerID="7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e" Jul 09 17:42:59.419850 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.419805 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e"} err="failed to get container status \"7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e\": rpc error: code = NotFound desc = could not find container \"7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e\": container with ID starting with 7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e not found: ID does not exist" Jul 09 17:42:59.419850 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.419820 2571 scope.go:117] "RemoveContainer" containerID="1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd" Jul 09 17:42:59.420009 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:42:59.419993 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd\": container with ID starting with 1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd not found: ID does not exist" containerID="1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd" Jul 09 17:42:59.420043 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.420014 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd"} err="failed to get container status \"1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd\": rpc error: code = NotFound desc = could not find container \"1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd\": container with ID starting with 1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd not found: ID does not exist" Jul 09 17:42:59.420043 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.420026 2571 scope.go:117] "RemoveContainer" containerID="d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03" Jul 09 17:42:59.420227 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:42:59.420211 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03\": container with ID starting with d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03 not found: ID does not exist" containerID="d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03" Jul 09 17:42:59.420291 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.420230 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03"} err="failed to get container status \"d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03\": rpc error: code = NotFound desc = could not find container \"d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03\": container with ID starting with d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03 not found: ID does not exist" Jul 09 17:42:59.420291 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.420241 2571 scope.go:117] "RemoveContainer" containerID="0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf" Jul 09 17:42:59.420453 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:42:59.420438 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf\": container with ID starting with 0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf not found: ID does not exist" containerID="0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf" Jul 09 17:42:59.420491 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.420455 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf"} err="failed to get container status \"0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf\": rpc error: code = NotFound desc = could not find container \"0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf\": container with ID starting with 0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf not found: ID does not exist" Jul 09 17:42:59.420491 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.420467 2571 scope.go:117] "RemoveContainer" containerID="801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4" Jul 09 17:42:59.420678 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:42:59.420665 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4\": container with ID starting with 801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4 not found: ID does not exist" containerID="801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4" Jul 09 17:42:59.420711 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.420681 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4"} err="failed to get container status \"801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4\": rpc error: code = NotFound desc = could not find container \"801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4\": container with ID starting with 801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4 not found: ID does not exist" Jul 09 17:42:59.420711 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.420693 2571 scope.go:117] "RemoveContainer" containerID="79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2" Jul 09 17:42:59.420882 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.420867 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2"} err="failed to get container status \"79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2\": rpc error: code = NotFound desc = could not find container \"79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2\": container with ID starting with 79034e6424eb62cbfff899b75a1d5649d6bc438493bdb56d6553db8471fb84f2 not found: ID does not exist" Jul 09 17:42:59.420920 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.420882 2571 scope.go:117] "RemoveContainer" containerID="7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf" Jul 09 17:42:59.421072 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.421056 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf"} err="failed to get container status \"7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf\": rpc error: code = NotFound desc = could not find container \"7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf\": container with ID starting with 7979cd36e8dbd7a3b3fa859c44bd7257920ab8ef7e494c8d1dffa82168961daf not found: ID does not exist" Jul 09 17:42:59.421120 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.421072 2571 scope.go:117] "RemoveContainer" containerID="7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e" Jul 09 17:42:59.421252 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.421237 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e"} err="failed to get container status \"7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e\": rpc error: code = NotFound desc = could not find container \"7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e\": container with ID starting with 7fc6a4dcc96bd4ffcb2c307871907a2bbcfff7af9492ea76e2f0d563d8e3655e not found: ID does not exist" Jul 09 17:42:59.421311 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.421253 2571 scope.go:117] "RemoveContainer" containerID="1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd" Jul 09 17:42:59.421489 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.421471 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd"} err="failed to get container status \"1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd\": rpc error: code = NotFound desc = could not find container \"1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd\": container with ID starting with 1a0b370fd58986a79ba35b46697413ad84e2dedeb65132d0b3ee305c39e05edd not found: ID does not exist" Jul 09 17:42:59.421527 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.421490 2571 scope.go:117] "RemoveContainer" containerID="d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03" Jul 09 17:42:59.421700 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.421682 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03"} err="failed to get container status \"d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03\": rpc error: code = NotFound desc = could not find container \"d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03\": container with ID starting with d76b896f446baceea9c0234f9ccf4f8298a35c2a23dab9edc7bd9cbe9a347c03 not found: ID does not exist" Jul 09 17:42:59.421749 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.421701 2571 scope.go:117] "RemoveContainer" containerID="0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf" Jul 09 17:42:59.421897 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.421874 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf"} err="failed to get container status \"0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf\": rpc error: code = NotFound desc = could not find container \"0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf\": container with ID starting with 0cc1a273205a9a07631841420d5a6bbf1c68bd3ab01097c4d002c3bb98c03caf not found: ID does not exist" Jul 09 17:42:59.421933 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.421898 2571 scope.go:117] "RemoveContainer" containerID="801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4" Jul 09 17:42:59.422095 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.422080 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4"} err="failed to get container status \"801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4\": rpc error: code = NotFound desc = could not find container \"801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4\": container with ID starting with 801372a1f9f18004de2514c68bf8fc020e218bb2568d00d870f568c9d2b5dcc4 not found: ID does not exist" Jul 09 17:42:59.490487 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490461 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490487 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490483 2571 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-metrics-client-ca\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490492 2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-main-db\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490503 2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-tls-assets\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490511 2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-out\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490518 2571 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-cluster-tls-config\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490526 2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-web-config\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490534 2571 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-config-volume\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490542 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490551 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490559 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fxxn\" (UniqueName: \"kubernetes.io/projected/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-kube-api-access-5fxxn\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490569 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-secret-alertmanager-main-tls\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.490590 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.490577 2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8f57a9-7636-4ab5-86cd-39fea28b88dc-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:42:59.699246 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.699192 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:42:59.707718 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.707691 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:42:59.733396 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733365 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:42:59.733655 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733643 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" containerName="registry" Jul 09 17:42:59.733697 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733656 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" containerName="registry" Jul 09 17:42:59.733697 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733665 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="prom-label-proxy" Jul 09 17:42:59.733697 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733671 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="prom-label-proxy" Jul 09 17:42:59.733697 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733679 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy-metric" Jul 09 17:42:59.733697 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733686 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy-metric" Jul 09 17:42:59.733697 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733694 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5422e158-97d6-4d3a-a189-a790daa14a6a" containerName="console" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733700 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5422e158-97d6-4d3a-a189-a790daa14a6a" containerName="console" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733708 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="init-config-reloader" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733713 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="init-config-reloader" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733720 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="alertmanager" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733725 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="alertmanager" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733733 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="config-reloader" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733738 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="config-reloader" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733747 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy-web" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733752 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy-web" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733761 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733767 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733813 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="127cf5c0-21b8-4fb4-a147-a12d89d6ef7a" containerName="registry" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733820 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="config-reloader" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733827 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy-web" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733834 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733840 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5422e158-97d6-4d3a-a189-a790daa14a6a" containerName="console" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733846 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="kube-rbac-proxy-metric" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733851 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="prom-label-proxy" Jul 09 17:42:59.733859 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.733857 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" containerName="alertmanager" Jul 09 17:42:59.738907 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.738887 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.741433 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.741404 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 17:42:59.741433 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.741422 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 17:42:59.741584 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.741438 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 17:42:59.741584 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.741484 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 17:42:59.741584 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.741505 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 17:42:59.741980 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.741965 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xfd85\"" Jul 09 17:42:59.742082 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.742055 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 17:42:59.742082 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.742065 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 17:42:59.742251 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.742194 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 17:42:59.747517 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.747499 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 17:42:59.755673 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.755646 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:42:59.894258 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894209 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894258 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894257 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28af0edb-e52f-4f2f-8413-028d03f56308-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894502 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894300 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28af0edb-e52f-4f2f-8413-028d03f56308-config-out\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894502 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894375 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-web-config\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894502 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894401 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28af0edb-e52f-4f2f-8413-028d03f56308-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894502 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894421 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wfqp\" (UniqueName: \"kubernetes.io/projected/28af0edb-e52f-4f2f-8413-028d03f56308-kube-api-access-7wfqp\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894502 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894437 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28af0edb-e52f-4f2f-8413-028d03f56308-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894502 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894462 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894502 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894478 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894502 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894496 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894770 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894517 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28af0edb-e52f-4f2f-8413-028d03f56308-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894770 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894533 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-config-volume\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.894770 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.894553 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995239 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995153 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28af0edb-e52f-4f2f-8413-028d03f56308-config-out\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995239 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995200 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-web-config\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995483 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995338 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28af0edb-e52f-4f2f-8413-028d03f56308-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995483 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995384 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wfqp\" (UniqueName: \"kubernetes.io/projected/28af0edb-e52f-4f2f-8413-028d03f56308-kube-api-access-7wfqp\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995483 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995419 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28af0edb-e52f-4f2f-8413-028d03f56308-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995483 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995466 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995706 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995495 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995706 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995529 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995706 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995554 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28af0edb-e52f-4f2f-8413-028d03f56308-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995706 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995581 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-config-volume\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995706 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995616 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995706 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995684 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.995986 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995709 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28af0edb-e52f-4f2f-8413-028d03f56308-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.996713 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.995714 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28af0edb-e52f-4f2f-8413-028d03f56308-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.997855 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.996998 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28af0edb-e52f-4f2f-8413-028d03f56308-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.997855 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.997736 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28af0edb-e52f-4f2f-8413-028d03f56308-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.998004 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.997918 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28af0edb-e52f-4f2f-8413-028d03f56308-config-out\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.998246 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.998218 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.998368 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.998304 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-web-config\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.998441 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.998403 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.999453 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.999337 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28af0edb-e52f-4f2f-8413-028d03f56308-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:42:59.999826 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:42:59.999800 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:43:00.001255 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:00.001232 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:43:00.002508 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:00.001697 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:43:00.004252 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:00.002959 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28af0edb-e52f-4f2f-8413-028d03f56308-config-volume\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:43:00.004252 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:00.003978 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wfqp\" (UniqueName: \"kubernetes.io/projected/28af0edb-e52f-4f2f-8413-028d03f56308-kube-api-access-7wfqp\") pod \"alertmanager-main-0\" (UID: \"28af0edb-e52f-4f2f-8413-028d03f56308\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:43:00.048786 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:00.048754 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:43:00.197417 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:00.197380 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:43:00.201505 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:43:00.201470 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28af0edb_e52f_4f2f_8413_028d03f56308.slice/crio-cb25b7b7ae561ab3d832f3e5d3c38ad7c3d5a336477e7517abfcfd067e9a9098 WatchSource:0}: Error finding container cb25b7b7ae561ab3d832f3e5d3c38ad7c3d5a336477e7517abfcfd067e9a9098: Status 404 returned error can't find the container with id cb25b7b7ae561ab3d832f3e5d3c38ad7c3d5a336477e7517abfcfd067e9a9098 Jul 09 17:43:00.368047 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:00.368014 2571 generic.go:356] "Generic (PLEG): container finished" podID="28af0edb-e52f-4f2f-8413-028d03f56308" containerID="962c499b30cf4d4962031cfadedd352753a13826c39c3e278856c24494005ea2" exitCode=0 Jul 09 17:43:00.368454 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:00.368108 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28af0edb-e52f-4f2f-8413-028d03f56308","Type":"ContainerDied","Data":"962c499b30cf4d4962031cfadedd352753a13826c39c3e278856c24494005ea2"} Jul 09 17:43:00.368454 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:00.368178 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28af0edb-e52f-4f2f-8413-028d03f56308","Type":"ContainerStarted","Data":"cb25b7b7ae561ab3d832f3e5d3c38ad7c3d5a336477e7517abfcfd067e9a9098"} Jul 09 17:43:00.479432 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:00.479400 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f8f57a9-7636-4ab5-86cd-39fea28b88dc" path="/var/lib/kubelet/pods/1f8f57a9-7636-4ab5-86cd-39fea28b88dc/volumes" Jul 09 17:43:01.376989 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:01.376953 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28af0edb-e52f-4f2f-8413-028d03f56308","Type":"ContainerStarted","Data":"6e9013cbfb04038c96d163182e7ae0f271c3391c91f45fd836f90c6b94b54a04"} Jul 09 17:43:01.377446 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:01.377031 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28af0edb-e52f-4f2f-8413-028d03f56308","Type":"ContainerStarted","Data":"c299faf2e4b3f748229b9a6d115c864a41026f32a40cf50cb6e4ba5e50f641f4"} Jul 09 17:43:01.377446 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:01.377081 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28af0edb-e52f-4f2f-8413-028d03f56308","Type":"ContainerStarted","Data":"4211f7c80c3e2677600e1dde15710c3634c247d2658f719078248108ddf19ae4"} Jul 09 17:43:01.377446 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:01.377125 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28af0edb-e52f-4f2f-8413-028d03f56308","Type":"ContainerStarted","Data":"392243e31f4205d662c257a63a57287d2f31c76afb6500a342f391d368a03a38"} Jul 09 17:43:01.377446 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:01.377172 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28af0edb-e52f-4f2f-8413-028d03f56308","Type":"ContainerStarted","Data":"7809f33874d781abb0e576f9decc8b57bb059efae1157891689d0837977687c5"} Jul 09 17:43:01.377446 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:01.377211 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28af0edb-e52f-4f2f-8413-028d03f56308","Type":"ContainerStarted","Data":"748e354c63efb863f16020f476068e44cd795bbbd4616b417c4cf9de0321ef5e"} Jul 09 17:43:01.408703 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:01.408653 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.394150156 podStartE2EDuration="2.408634731s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.484575ms" isStatefulPod=false podCreationTimestamp="2026-07-09 17:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:43:01.4057762 +0000 UTC m=+197.491899667" watchObservedRunningTime="2026-07-09 17:43:01.408634731 +0000 UTC m=+197.494758196" Jul 09 17:43:02.256768 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.256727 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:43:02.257163 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.257129 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="prometheus" containerID="cri-o://f5a3c2a4e502328162b39daf667eecea7e855132568eba15cddcfbe77f3f2b84" gracePeriod=600 Jul 09 17:43:02.257163 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.257158 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="thanos-sidecar" containerID="cri-o://0d28336cfda212693442a8f074796070e5dbbf5b8151acbdf5ebba07c502c162" gracePeriod=600 Jul 09 17:43:02.257433 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.257138 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="config-reloader" containerID="cri-o://8c18239f1b2f7e74166f468734093fe9185ed74d24c307caec207e633d225a16" gracePeriod=600 Jul 09 17:43:02.257433 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.257294 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy" containerID="cri-o://1e114bc12bfb4492fe41a627b9a6cee856be114368e79b2c10936f507e057ed4" gracePeriod=600 Jul 09 17:43:02.257433 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.257323 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy-web" containerID="cri-o://d4c4e2d6a9b704e13c3a65ecb0f629a9d121b2f9db65523b712b20516b6b30db" gracePeriod=600 Jul 09 17:43:02.257433 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.257370 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8d83ee3562d9d2eba7adbff19d948af25b2078f9672c4ba1d4a78af791618e09" gracePeriod=600 Jul 09 17:43:02.385943 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.385914 2571 generic.go:356] "Generic (PLEG): container finished" podID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerID="8d83ee3562d9d2eba7adbff19d948af25b2078f9672c4ba1d4a78af791618e09" exitCode=0 Jul 09 17:43:02.385943 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.385942 2571 generic.go:356] "Generic (PLEG): container finished" podID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerID="1e114bc12bfb4492fe41a627b9a6cee856be114368e79b2c10936f507e057ed4" exitCode=0 Jul 09 17:43:02.386390 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.385952 2571 generic.go:356] "Generic (PLEG): container finished" podID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerID="d4c4e2d6a9b704e13c3a65ecb0f629a9d121b2f9db65523b712b20516b6b30db" exitCode=0 Jul 09 17:43:02.386390 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.385959 2571 generic.go:356] "Generic (PLEG): container finished" podID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerID="0d28336cfda212693442a8f074796070e5dbbf5b8151acbdf5ebba07c502c162" exitCode=0 Jul 09 17:43:02.386390 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.385964 2571 generic.go:356] "Generic (PLEG): container finished" podID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerID="8c18239f1b2f7e74166f468734093fe9185ed74d24c307caec207e633d225a16" exitCode=0 Jul 09 17:43:02.386390 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.385969 2571 generic.go:356] "Generic (PLEG): container finished" podID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerID="f5a3c2a4e502328162b39daf667eecea7e855132568eba15cddcfbe77f3f2b84" exitCode=0 Jul 09 17:43:02.386390 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.385985 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerDied","Data":"8d83ee3562d9d2eba7adbff19d948af25b2078f9672c4ba1d4a78af791618e09"} Jul 09 17:43:02.386390 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.386050 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerDied","Data":"1e114bc12bfb4492fe41a627b9a6cee856be114368e79b2c10936f507e057ed4"} Jul 09 17:43:02.386390 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.386076 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerDied","Data":"d4c4e2d6a9b704e13c3a65ecb0f629a9d121b2f9db65523b712b20516b6b30db"} Jul 09 17:43:02.386390 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.386101 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerDied","Data":"0d28336cfda212693442a8f074796070e5dbbf5b8151acbdf5ebba07c502c162"} Jul 09 17:43:02.386390 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.386124 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerDied","Data":"8c18239f1b2f7e74166f468734093fe9185ed74d24c307caec207e633d225a16"} Jul 09 17:43:02.386390 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.386146 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerDied","Data":"f5a3c2a4e502328162b39daf667eecea7e855132568eba15cddcfbe77f3f2b84"} Jul 09 17:43:02.511738 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.511682 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:02.620681 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.620647 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-web-config\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.620681 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.620685 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-2\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.620904 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.620709 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-db\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.620904 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.620740 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-tls\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.620904 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.620864 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621046 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.620916 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-kube-rbac-proxy\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621046 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.620945 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-1\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621046 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.620980 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-0\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621046 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621008 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-trusted-ca-bundle\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621237 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621037 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:43:02.621237 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621050 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-metrics-client-ca\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621237 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621110 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621237 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621214 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tktzm\" (UniqueName: \"kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-kube-api-access-tktzm\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621528 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621245 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-thanos-prometheus-http-client-file\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621528 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621302 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:43:02.621528 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621311 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-kubelet-serving-ca-bundle\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621528 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621372 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-serving-certs-ca-bundle\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621528 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621412 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-grpc-tls\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621528 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621436 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-tls-assets\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621528 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621442 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:43:02.621528 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621471 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-metrics-client-certs\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621528 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621518 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621546 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config-out\") pod \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\" (UID: \"b4059f1d-a41c-4a56-960b-03984ad6a0e4\") " Jul 09 17:43:02.621958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621585 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:43:02.621958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621705 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:43:02.621958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621864 2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.621958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621885 2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-db\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.621958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621901 2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.621958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621916 2571 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-metrics-client-ca\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.621958 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.621932 2571 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.622554 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.622481 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:43:02.622779 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.622760 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:43:02.623375 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.623348 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config" (OuterVolumeSpecName: "config") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:43:02.623863 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.623382 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:43:02.623863 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.623422 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:43:02.623863 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.623777 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:43:02.624067 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.623993 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:43:02.624331 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.624304 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config-out" (OuterVolumeSpecName: "config-out") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:43:02.624927 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.624900 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:43:02.625465 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.625440 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-kube-api-access-tktzm" (OuterVolumeSpecName: "kube-api-access-tktzm") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "kube-api-access-tktzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:43:02.625705 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.625675 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:43:02.625810 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.625735 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:43:02.625892 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.625872 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:43:02.626124 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.626105 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:43:02.635530 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.635510 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-web-config" (OuterVolumeSpecName: "web-config") pod "b4059f1d-a41c-4a56-960b-03984ad6a0e4" (UID: "b4059f1d-a41c-4a56-960b-03984ad6a0e4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:43:02.722492 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722459 2571 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-grpc-tls\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722492 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722487 2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-tls-assets\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722501 2571 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-metrics-client-certs\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722517 2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722529 2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config-out\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722540 2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-web-config\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722552 2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722563 2571 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-config\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722575 2571 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-kube-rbac-proxy\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722586 2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722598 2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722611 2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722625 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tktzm\" (UniqueName: \"kubernetes.io/projected/b4059f1d-a41c-4a56-960b-03984ad6a0e4-kube-api-access-tktzm\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722640 2571 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b4059f1d-a41c-4a56-960b-03984ad6a0e4-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:02.722686 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:02.722655 2571 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4059f1d-a41c-4a56-960b-03984ad6a0e4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:43:03.393056 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.393024 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.393485 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.393024 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b4059f1d-a41c-4a56-960b-03984ad6a0e4","Type":"ContainerDied","Data":"d1d7be2278cfda8131eccaf95375ac9b660aa575544cdee07536fab84182cf3b"} Jul 09 17:43:03.393485 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.393169 2571 scope.go:117] "RemoveContainer" containerID="8d83ee3562d9d2eba7adbff19d948af25b2078f9672c4ba1d4a78af791618e09" Jul 09 17:43:03.402052 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.402015 2571 scope.go:117] "RemoveContainer" containerID="1e114bc12bfb4492fe41a627b9a6cee856be114368e79b2c10936f507e057ed4" Jul 09 17:43:03.409661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.409645 2571 scope.go:117] "RemoveContainer" containerID="d4c4e2d6a9b704e13c3a65ecb0f629a9d121b2f9db65523b712b20516b6b30db" Jul 09 17:43:03.416207 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.416190 2571 scope.go:117] "RemoveContainer" containerID="0d28336cfda212693442a8f074796070e5dbbf5b8151acbdf5ebba07c502c162" Jul 09 17:43:03.418933 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.418909 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:43:03.423603 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.423585 2571 scope.go:117] "RemoveContainer" containerID="8c18239f1b2f7e74166f468734093fe9185ed74d24c307caec207e633d225a16" Jul 09 17:43:03.423940 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.423914 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:43:03.430076 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.430054 2571 scope.go:117] "RemoveContainer" containerID="f5a3c2a4e502328162b39daf667eecea7e855132568eba15cddcfbe77f3f2b84" Jul 09 17:43:03.436784 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.436767 2571 scope.go:117] "RemoveContainer" containerID="17a88c13c2d72451dd5bdb9874b73c841ee578baa95f49391d7b3eaeee33cca9" Jul 09 17:43:03.449448 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449427 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:43:03.449744 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449732 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="config-reloader" Jul 09 17:43:03.449792 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449745 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="config-reloader" Jul 09 17:43:03.449792 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449756 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="prometheus" Jul 09 17:43:03.449792 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449762 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="prometheus" Jul 09 17:43:03.449792 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449771 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="thanos-sidecar" Jul 09 17:43:03.449792 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449776 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="thanos-sidecar" Jul 09 17:43:03.449792 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449783 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="init-config-reloader" Jul 09 17:43:03.449792 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449788 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="init-config-reloader" Jul 09 17:43:03.449792 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449793 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy-thanos" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449798 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy-thanos" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449809 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449814 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449826 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy-web" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449833 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy-web" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449877 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449884 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="config-reloader" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449891 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy-web" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449896 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="thanos-sidecar" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449902 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="prometheus" Jul 09 17:43:03.450023 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.449909 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" containerName="kube-rbac-proxy-thanos" Jul 09 17:43:03.453739 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.453720 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.456506 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.456483 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 17:43:03.456607 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.456539 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 17:43:03.456607 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.456550 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 17:43:03.456722 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.456613 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 17:43:03.456838 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.456810 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 17:43:03.456976 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.456872 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 17:43:03.457158 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.457110 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 17:43:03.457158 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.457115 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 17:43:03.457373 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.457209 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 17:43:03.457373 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.457229 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-76aqvjiju0qrf\"" Jul 09 17:43:03.457952 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.457909 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-77p6b\"" Jul 09 17:43:03.457952 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.457928 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 17:43:03.458097 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.457992 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 17:43:03.458097 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.457908 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 17:43:03.458237 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.458220 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 17:43:03.461282 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.461243 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 17:43:03.463689 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.463661 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 17:43:03.469025 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.469002 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:43:03.628716 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.628684 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.628939 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.628727 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.628939 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.628746 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.628939 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.628821 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.628939 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.628856 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.628939 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.628888 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629098 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.628939 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629098 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.628962 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629098 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.628988 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629098 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629006 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629098 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629024 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-config\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629098 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629046 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629098 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629071 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629098 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629087 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/460c63ca-2179-47a8-b97a-8c88bf73c7d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629386 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629104 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629386 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629158 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9btq\" (UniqueName: \"kubernetes.io/projected/460c63ca-2179-47a8-b97a-8c88bf73c7d3-kube-api-access-b9btq\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629386 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629187 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629386 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629209 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/460c63ca-2179-47a8-b97a-8c88bf73c7d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629386 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629225 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.629386 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.629257 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.729815 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.729787 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.729964 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.729823 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/460c63ca-2179-47a8-b97a-8c88bf73c7d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.729964 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.729842 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.729964 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.729874 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.729964 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.729911 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.729964 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.729948 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730190 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730088 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730190 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730150 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730190 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730182 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730380 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730225 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730380 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730258 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730380 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730307 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730380 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730338 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730380 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730362 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730628 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730388 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-config\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730628 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730400 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730628 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730414 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730628 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730455 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730628 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730484 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/460c63ca-2179-47a8-b97a-8c88bf73c7d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730628 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730513 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730628 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730551 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9btq\" (UniqueName: \"kubernetes.io/projected/460c63ca-2179-47a8-b97a-8c88bf73c7d3-kube-api-access-b9btq\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730982 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730722 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730982 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730881 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.730982 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730978 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.731942 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.731913 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.732796 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.732755 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.734419 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.730511 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.734419 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.733605 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.734419 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.733807 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.734419 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.733943 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-config\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.734419 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.733946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/460c63ca-2179-47a8-b97a-8c88bf73c7d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.735107 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.735038 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/460c63ca-2179-47a8-b97a-8c88bf73c7d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.735169 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.735112 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.737204 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.735297 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.737204 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.735463 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.737204 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.736011 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.737204 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.736744 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.737204 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.736952 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/460c63ca-2179-47a8-b97a-8c88bf73c7d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.739360 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.739333 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9btq\" (UniqueName: \"kubernetes.io/projected/460c63ca-2179-47a8-b97a-8c88bf73c7d3-kube-api-access-b9btq\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.741526 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.741507 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/460c63ca-2179-47a8-b97a-8c88bf73c7d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"460c63ca-2179-47a8-b97a-8c88bf73c7d3\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.763692 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.763658 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:43:03.898414 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:03.898339 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:43:03.902122 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:43:03.902093 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod460c63ca_2179_47a8_b97a_8c88bf73c7d3.slice/crio-482e29146f6cf182ab25b0a4de065e34561ddf1a77acd7023f50d71e80b41334 WatchSource:0}: Error finding container 482e29146f6cf182ab25b0a4de065e34561ddf1a77acd7023f50d71e80b41334: Status 404 returned error can't find the container with id 482e29146f6cf182ab25b0a4de065e34561ddf1a77acd7023f50d71e80b41334 Jul 09 17:43:04.399345 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:04.399315 2571 generic.go:356] "Generic (PLEG): container finished" podID="460c63ca-2179-47a8-b97a-8c88bf73c7d3" containerID="d655bd3cec3ddd4759e701e91531a46883ecaa84a374333923e67429919e6189" exitCode=0 Jul 09 17:43:04.399775 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:04.399393 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"460c63ca-2179-47a8-b97a-8c88bf73c7d3","Type":"ContainerDied","Data":"d655bd3cec3ddd4759e701e91531a46883ecaa84a374333923e67429919e6189"} Jul 09 17:43:04.399775 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:04.399481 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"460c63ca-2179-47a8-b97a-8c88bf73c7d3","Type":"ContainerStarted","Data":"482e29146f6cf182ab25b0a4de065e34561ddf1a77acd7023f50d71e80b41334"} Jul 09 17:43:04.478336 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:04.478308 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4059f1d-a41c-4a56-960b-03984ad6a0e4" path="/var/lib/kubelet/pods/b4059f1d-a41c-4a56-960b-03984ad6a0e4/volumes" Jul 09 17:43:05.406479 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:05.406444 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"460c63ca-2179-47a8-b97a-8c88bf73c7d3","Type":"ContainerStarted","Data":"27c514c5294c2aed3deeb65682ff940756b8197b6bef156dc2453104e1099a83"} Jul 09 17:43:05.406877 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:05.406506 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"460c63ca-2179-47a8-b97a-8c88bf73c7d3","Type":"ContainerStarted","Data":"a792d5a442f37cc3fe182c5399387cbe76e0ebdc8e51bf37c75ab2150e662540"} Jul 09 17:43:05.406877 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:05.406533 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"460c63ca-2179-47a8-b97a-8c88bf73c7d3","Type":"ContainerStarted","Data":"080c31f00364d642e4a6adaccb888666a53a2e2c2cb4a1c5444668900a162f80"} Jul 09 17:43:05.406877 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:05.406559 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"460c63ca-2179-47a8-b97a-8c88bf73c7d3","Type":"ContainerStarted","Data":"00187c24e0ff65828f5abe4897e20e05a7e2e7cd5f14e485e8b88ca8f630814e"} Jul 09 17:43:05.406877 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:05.406582 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"460c63ca-2179-47a8-b97a-8c88bf73c7d3","Type":"ContainerStarted","Data":"07f9e9114a6688e0150226a0e2bf1413b1f1ea24b4f564513d194e682ee68bdd"} Jul 09 17:43:05.406877 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:05.406604 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"460c63ca-2179-47a8-b97a-8c88bf73c7d3","Type":"ContainerStarted","Data":"ccd6095a6c99d7720ce7c6468e67770e6750307227cd68bd0f09f4fb27913517"} Jul 09 17:43:05.436410 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:05.436352 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.414893855 podStartE2EDuration="2.436331786s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.437931ms" isStatefulPod=false podCreationTimestamp="2026-07-09 17:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:43:05.434683282 +0000 UTC m=+201.520806770" watchObservedRunningTime="2026-07-09 17:43:05.436331786 +0000 UTC m=+201.522455253" Jul 09 17:43:08.763965 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:43:08.763930 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:44:03.764517 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:44:03.764480 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:44:03.779609 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:44:03.779581 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:44:04.654888 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:44:04.654862 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:44:44.290833 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:44:44.290801 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:44:44.296196 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:44:44.296176 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:44:44.298149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:44:44.298127 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:44:44.301853 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:44:44.301836 2571 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 17:44:44.304224 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:44:44.304205 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:47:29.624585 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.624551 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-2q7j4"] Jul 09 17:47:29.628100 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.628076 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-2q7j4" Jul 09 17:47:29.632737 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.632711 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 09 17:47:29.632874 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.632850 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-pt4wf\"" Jul 09 17:47:29.633726 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.633708 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 09 17:47:29.633822 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.633758 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 09 17:47:29.639814 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.639790 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-2q7j4"] Jul 09 17:47:29.808149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.808116 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76d8ab87-2c47-4053-a99d-dd38c95741fa-model-cache\") pod \"s3-init-2q7j4\" (UID: \"76d8ab87-2c47-4053-a99d-dd38c95741fa\") " pod="kserve/s3-init-2q7j4" Jul 09 17:47:29.808332 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.808179 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj7nw\" (UniqueName: \"kubernetes.io/projected/76d8ab87-2c47-4053-a99d-dd38c95741fa-kube-api-access-xj7nw\") pod \"s3-init-2q7j4\" (UID: \"76d8ab87-2c47-4053-a99d-dd38c95741fa\") " pod="kserve/s3-init-2q7j4" Jul 09 17:47:29.909044 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.908962 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj7nw\" (UniqueName: \"kubernetes.io/projected/76d8ab87-2c47-4053-a99d-dd38c95741fa-kube-api-access-xj7nw\") pod \"s3-init-2q7j4\" (UID: \"76d8ab87-2c47-4053-a99d-dd38c95741fa\") " pod="kserve/s3-init-2q7j4" Jul 09 17:47:29.909044 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.909013 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76d8ab87-2c47-4053-a99d-dd38c95741fa-model-cache\") pod \"s3-init-2q7j4\" (UID: \"76d8ab87-2c47-4053-a99d-dd38c95741fa\") " pod="kserve/s3-init-2q7j4" Jul 09 17:47:29.909385 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.909368 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76d8ab87-2c47-4053-a99d-dd38c95741fa-model-cache\") pod \"s3-init-2q7j4\" (UID: \"76d8ab87-2c47-4053-a99d-dd38c95741fa\") " pod="kserve/s3-init-2q7j4" Jul 09 17:47:29.917677 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.917652 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj7nw\" (UniqueName: \"kubernetes.io/projected/76d8ab87-2c47-4053-a99d-dd38c95741fa-kube-api-access-xj7nw\") pod \"s3-init-2q7j4\" (UID: \"76d8ab87-2c47-4053-a99d-dd38c95741fa\") " pod="kserve/s3-init-2q7j4" Jul 09 17:47:29.937646 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:29.937620 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-2q7j4" Jul 09 17:47:30.067638 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:30.067606 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-2q7j4"] Jul 09 17:47:30.070194 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:47:30.070166 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76d8ab87_2c47_4053_a99d_dd38c95741fa.slice/crio-5aac5bf47c247707681033b2722df4bb995e2b64aa87cf5f29c29ca7a3b8a4b4 WatchSource:0}: Error finding container 5aac5bf47c247707681033b2722df4bb995e2b64aa87cf5f29c29ca7a3b8a4b4: Status 404 returned error can't find the container with id 5aac5bf47c247707681033b2722df4bb995e2b64aa87cf5f29c29ca7a3b8a4b4 Jul 09 17:47:30.072036 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:30.072020 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:47:30.422685 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:30.422649 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2q7j4" event={"ID":"76d8ab87-2c47-4053-a99d-dd38c95741fa","Type":"ContainerStarted","Data":"5aac5bf47c247707681033b2722df4bb995e2b64aa87cf5f29c29ca7a3b8a4b4"} Jul 09 17:47:34.439407 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:34.439368 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2q7j4" event={"ID":"76d8ab87-2c47-4053-a99d-dd38c95741fa","Type":"ContainerStarted","Data":"d702fcb28e52e49dd223996ab11aa45b149e5b585ee2af578de96dd3b1068baa"} Jul 09 17:47:36.449178 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:36.449141 2571 generic.go:356] "Generic (PLEG): container finished" podID="76d8ab87-2c47-4053-a99d-dd38c95741fa" containerID="d702fcb28e52e49dd223996ab11aa45b149e5b585ee2af578de96dd3b1068baa" exitCode=0 Jul 09 17:47:36.449564 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:36.449221 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2q7j4" event={"ID":"76d8ab87-2c47-4053-a99d-dd38c95741fa","Type":"ContainerDied","Data":"d702fcb28e52e49dd223996ab11aa45b149e5b585ee2af578de96dd3b1068baa"} Jul 09 17:47:41.474042 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:41.474007 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2q7j4" event={"ID":"76d8ab87-2c47-4053-a99d-dd38c95741fa","Type":"ContainerStarted","Data":"0c9d228bdb6923c9e01e82b57858b2d1b57f449aa0e8d7bccbea636f6946b632"} Jul 09 17:47:41.493843 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:47:41.493777 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-2q7j4" podStartSLOduration=2.54646172 podStartE2EDuration="12.493757087s" totalImagesPullingTime="8.631986238s" totalInitContainerRuntime="1.315309129s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:47:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:47:41.493557096 +0000 UTC m=+477.579680562" watchObservedRunningTime="2026-07-09 17:47:41.493757087 +0000 UTC m=+477.579880565" Jul 09 17:48:46.724263 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:46.724230 2571 generic.go:356] "Generic (PLEG): container finished" podID="76d8ab87-2c47-4053-a99d-dd38c95741fa" containerID="0c9d228bdb6923c9e01e82b57858b2d1b57f449aa0e8d7bccbea636f6946b632" exitCode=0 Jul 09 17:48:46.724263 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:46.724291 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2q7j4" event={"ID":"76d8ab87-2c47-4053-a99d-dd38c95741fa","Type":"ContainerDied","Data":"0c9d228bdb6923c9e01e82b57858b2d1b57f449aa0e8d7bccbea636f6946b632"} Jul 09 17:48:47.850555 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:47.850522 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-2q7j4" Jul 09 17:48:47.939405 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:47.939376 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xj7nw\" (UniqueName: \"kubernetes.io/projected/76d8ab87-2c47-4053-a99d-dd38c95741fa-kube-api-access-xj7nw\") pod \"76d8ab87-2c47-4053-a99d-dd38c95741fa\" (UID: \"76d8ab87-2c47-4053-a99d-dd38c95741fa\") " Jul 09 17:48:47.939589 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:47.939417 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76d8ab87-2c47-4053-a99d-dd38c95741fa-model-cache\") pod \"76d8ab87-2c47-4053-a99d-dd38c95741fa\" (UID: \"76d8ab87-2c47-4053-a99d-dd38c95741fa\") " Jul 09 17:48:47.941591 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:47.941557 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d8ab87-2c47-4053-a99d-dd38c95741fa-kube-api-access-xj7nw" (OuterVolumeSpecName: "kube-api-access-xj7nw") pod "76d8ab87-2c47-4053-a99d-dd38c95741fa" (UID: "76d8ab87-2c47-4053-a99d-dd38c95741fa"). InnerVolumeSpecName "kube-api-access-xj7nw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:48:47.957971 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:47.957945 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d8ab87-2c47-4053-a99d-dd38c95741fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "76d8ab87-2c47-4053-a99d-dd38c95741fa" (UID: "76d8ab87-2c47-4053-a99d-dd38c95741fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:48.040149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:48.040080 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xj7nw\" (UniqueName: \"kubernetes.io/projected/76d8ab87-2c47-4053-a99d-dd38c95741fa-kube-api-access-xj7nw\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:48:48.040149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:48.040105 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76d8ab87-2c47-4053-a99d-dd38c95741fa-model-cache\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:48:48.733336 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:48.733293 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2q7j4" event={"ID":"76d8ab87-2c47-4053-a99d-dd38c95741fa","Type":"ContainerDied","Data":"5aac5bf47c247707681033b2722df4bb995e2b64aa87cf5f29c29ca7a3b8a4b4"} Jul 09 17:48:48.733336 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:48.733340 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5aac5bf47c247707681033b2722df4bb995e2b64aa87cf5f29c29ca7a3b8a4b4" Jul 09 17:48:48.733571 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:48:48.733338 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-2q7j4" Jul 09 17:49:11.229690 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.229655 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf"] Jul 09 17:49:11.230160 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.230114 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d8ab87-2c47-4053-a99d-dd38c95741fa" containerName="s3-init" Jul 09 17:49:11.230160 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.230132 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d8ab87-2c47-4053-a99d-dd38c95741fa" containerName="s3-init" Jul 09 17:49:11.230160 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.230150 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d8ab87-2c47-4053-a99d-dd38c95741fa" containerName="download-hf-model" Jul 09 17:49:11.230160 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.230159 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d8ab87-2c47-4053-a99d-dd38c95741fa" containerName="download-hf-model" Jul 09 17:49:11.230405 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.230234 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d8ab87-2c47-4053-a99d-dd38c95741fa" containerName="s3-init" Jul 09 17:49:11.233285 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.233253 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.239431 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.239404 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 09 17:49:11.239569 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.239404 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 09 17:49:11.239569 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.239521 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-mspnj\"" Jul 09 17:49:11.240033 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.239404 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 09 17:49:11.240033 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.239404 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 09 17:49:11.240033 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.239640 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 09 17:49:11.240661 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.240643 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 09 17:49:11.252379 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.252358 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf"] Jul 09 17:49:11.325904 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.325875 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.326047 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.325913 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.326047 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.325955 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eab88989-e0f2-4690-857a-e9bb6e6d93c5-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.326047 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.326005 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjpth\" (UniqueName: \"kubernetes.io/projected/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-api-access-mjpth\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.326047 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.326029 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.426871 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.426834 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eab88989-e0f2-4690-857a-e9bb6e6d93c5-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.427024 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.426878 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjpth\" (UniqueName: \"kubernetes.io/projected/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-api-access-mjpth\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.427024 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.426900 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.427024 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.426968 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.427024 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.426991 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.427295 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.427242 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf"] Jul 09 17:49:11.427490 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.427469 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.427818 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.427800 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.427869 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.427824 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.429252 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.429234 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eab88989-e0f2-4690-857a-e9bb6e6d93c5-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.430986 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.430972 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.438578 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.438559 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 09 17:49:11.438842 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.438829 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 09 17:49:11.445552 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.445532 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjpth\" (UniqueName: \"kubernetes.io/projected/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-api-access-mjpth\") pod \"isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.453836 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.453811 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf"] Jul 09 17:49:11.527853 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.527772 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/423e843b-dfbe-4457-9ed1-6d9e3deec950-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.528012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.527890 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbpkq\" (UniqueName: \"kubernetes.io/projected/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-api-access-sbpkq\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.528012 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.527947 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.528144 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.528100 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/423e843b-dfbe-4457-9ed1-6d9e3deec950-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.528248 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.528207 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.543719 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.543691 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:11.628759 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.628726 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.628900 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.628786 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/423e843b-dfbe-4457-9ed1-6d9e3deec950-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.628900 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.628814 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.629105 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.629079 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/423e843b-dfbe-4457-9ed1-6d9e3deec950-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.629198 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.629156 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbpkq\" (UniqueName: \"kubernetes.io/projected/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-api-access-sbpkq\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.629621 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.629595 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.629784 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.629765 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/423e843b-dfbe-4457-9ed1-6d9e3deec950-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.630130 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.630108 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.631905 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.631872 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/423e843b-dfbe-4457-9ed1-6d9e3deec950-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.642752 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.642725 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbpkq\" (UniqueName: \"kubernetes.io/projected/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-api-access-sbpkq\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2nzjf\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.688316 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.688286 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf"] Jul 09 17:49:11.691904 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:49:11.691878 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab88989_e0f2_4690_857a_e9bb6e6d93c5.slice/crio-8127819ea34dd7c8858af0d70730dbe9de2da4e98802e3414b24a929f8ad22bb WatchSource:0}: Error finding container 8127819ea34dd7c8858af0d70730dbe9de2da4e98802e3414b24a929f8ad22bb: Status 404 returned error can't find the container with id 8127819ea34dd7c8858af0d70730dbe9de2da4e98802e3414b24a929f8ad22bb Jul 09 17:49:11.745250 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.745227 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:11.820853 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.820813 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" event={"ID":"eab88989-e0f2-4690-857a-e9bb6e6d93c5","Type":"ContainerStarted","Data":"bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53"} Jul 09 17:49:11.821006 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.820920 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" event={"ID":"eab88989-e0f2-4690-857a-e9bb6e6d93c5","Type":"ContainerStarted","Data":"8127819ea34dd7c8858af0d70730dbe9de2da4e98802e3414b24a929f8ad22bb"} Jul 09 17:49:11.826209 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.826177 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27"] Jul 09 17:49:11.830622 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.830600 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:11.833579 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.833555 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Jul 09 17:49:11.833698 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.833562 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Jul 09 17:49:11.844367 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.844261 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27"] Jul 09 17:49:11.891093 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.891069 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf"] Jul 09 17:49:11.893124 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:49:11.893095 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod423e843b_dfbe_4457_9ed1_6d9e3deec950.slice/crio-bb843531870e20bcc13b4210b69081ac45c66dd3168aeba9c86a0ecb9d972d4e WatchSource:0}: Error finding container bb843531870e20bcc13b4210b69081ac45c66dd3168aeba9c86a0ecb9d972d4e: Status 404 returned error can't find the container with id bb843531870e20bcc13b4210b69081ac45c66dd3168aeba9c86a0ecb9d972d4e Jul 09 17:49:11.932375 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.932354 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:11.932467 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.932387 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vg8g\" (UniqueName: \"kubernetes.io/projected/01e3411a-227b-4060-9de4-184750ea161d-kube-api-access-7vg8g\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:11.932546 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.932524 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01e3411a-227b-4060-9de4-184750ea161d-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:11.932641 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.932620 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01e3411a-227b-4060-9de4-184750ea161d-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:11.932733 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:11.932714 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.034340 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.034294 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.034521 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.034356 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vg8g\" (UniqueName: \"kubernetes.io/projected/01e3411a-227b-4060-9de4-184750ea161d-kube-api-access-7vg8g\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.034521 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.034385 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01e3411a-227b-4060-9de4-184750ea161d-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.034521 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.034421 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01e3411a-227b-4060-9de4-184750ea161d-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.034521 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.034481 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.034819 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.034789 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01e3411a-227b-4060-9de4-184750ea161d-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.035008 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.034977 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.035347 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.035325 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.036863 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.036838 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01e3411a-227b-4060-9de4-184750ea161d-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.045551 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.045501 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vg8g\" (UniqueName: \"kubernetes.io/projected/01e3411a-227b-4060-9de4-184750ea161d-kube-api-access-7vg8g\") pod \"isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.143255 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.143222 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:12.278355 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.278328 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27"] Jul 09 17:49:12.280754 ip-10-0-133-63 kubenswrapper[2571]: W0709 17:49:12.280726 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e3411a_227b_4060_9de4_184750ea161d.slice/crio-2b53a4ad722d03647daedce86b2657825e8ce4bfb5472a6f1eeac82749385d8c WatchSource:0}: Error finding container 2b53a4ad722d03647daedce86b2657825e8ce4bfb5472a6f1eeac82749385d8c: Status 404 returned error can't find the container with id 2b53a4ad722d03647daedce86b2657825e8ce4bfb5472a6f1eeac82749385d8c Jul 09 17:49:12.827380 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.827338 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" event={"ID":"01e3411a-227b-4060-9de4-184750ea161d","Type":"ContainerStarted","Data":"bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831"} Jul 09 17:49:12.827585 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.827418 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" event={"ID":"01e3411a-227b-4060-9de4-184750ea161d","Type":"ContainerStarted","Data":"2b53a4ad722d03647daedce86b2657825e8ce4bfb5472a6f1eeac82749385d8c"} Jul 09 17:49:12.828971 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.828939 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" event={"ID":"423e843b-dfbe-4457-9ed1-6d9e3deec950","Type":"ContainerStarted","Data":"e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815"} Jul 09 17:49:12.829070 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:12.828999 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" event={"ID":"423e843b-dfbe-4457-9ed1-6d9e3deec950","Type":"ContainerStarted","Data":"bb843531870e20bcc13b4210b69081ac45c66dd3168aeba9c86a0ecb9d972d4e"} Jul 09 17:49:15.842240 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:15.842150 2571 generic.go:356] "Generic (PLEG): container finished" podID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerID="bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53" exitCode=0 Jul 09 17:49:15.842240 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:15.842223 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" event={"ID":"eab88989-e0f2-4690-857a-e9bb6e6d93c5","Type":"ContainerDied","Data":"bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53"} Jul 09 17:49:15.843919 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:15.843895 2571 generic.go:356] "Generic (PLEG): container finished" podID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerID="e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815" exitCode=0 Jul 09 17:49:15.844032 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:15.843922 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" event={"ID":"423e843b-dfbe-4457-9ed1-6d9e3deec950","Type":"ContainerDied","Data":"e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815"} Jul 09 17:49:16.857138 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:16.857098 2571 generic.go:356] "Generic (PLEG): container finished" podID="01e3411a-227b-4060-9de4-184750ea161d" containerID="bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831" exitCode=0 Jul 09 17:49:16.857723 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:16.857163 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" event={"ID":"01e3411a-227b-4060-9de4-184750ea161d","Type":"ContainerDied","Data":"bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831"} Jul 09 17:49:42.014193 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:42.014095 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" event={"ID":"423e843b-dfbe-4457-9ed1-6d9e3deec950","Type":"ContainerStarted","Data":"a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760"} Jul 09 17:49:42.016327 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:42.016298 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" event={"ID":"01e3411a-227b-4060-9de4-184750ea161d","Type":"ContainerStarted","Data":"70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17"} Jul 09 17:49:43.026255 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:43.026169 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" event={"ID":"eab88989-e0f2-4690-857a-e9bb6e6d93c5","Type":"ContainerStarted","Data":"224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5"} Jul 09 17:49:44.323883 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:44.323856 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:49:44.325819 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:44.325797 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:49:44.330917 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:44.330899 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:49:44.332857 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:44.332840 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:49:45.037091 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.037052 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" event={"ID":"01e3411a-227b-4060-9de4-184750ea161d","Type":"ContainerStarted","Data":"882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c"} Jul 09 17:49:45.037295 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.037142 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:45.037295 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.037190 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:45.039500 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.039470 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" event={"ID":"eab88989-e0f2-4690-857a-e9bb6e6d93c5","Type":"ContainerStarted","Data":"929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f"} Jul 09 17:49:45.039770 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.039745 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Jul 09 17:49:45.040043 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.040020 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:45.040149 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.040134 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:45.041039 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.041003 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 09 17:49:45.042052 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.042025 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" event={"ID":"423e843b-dfbe-4457-9ed1-6d9e3deec950","Type":"ContainerStarted","Data":"62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1"} Jul 09 17:49:45.042191 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.042175 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:45.042336 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.042316 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:45.043116 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.043092 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Jul 09 17:49:45.072938 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.072894 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podStartSLOduration=3.146037918 podStartE2EDuration="34.072880098s" totalImagesPullingTime="27.214114469s" totalInitContainerRuntime="3.712727711s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:49:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:49:45.071790408 +0000 UTC m=+601.157913901" watchObservedRunningTime="2026-07-09 17:49:45.072880098 +0000 UTC m=+601.159003564" Jul 09 17:49:45.097354 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.097311 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podStartSLOduration=2.100428277 podStartE2EDuration="34.09729861s" totalImagesPullingTime="28.194183469s" totalInitContainerRuntime="3.802686864s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:49:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:49:45.095133827 +0000 UTC m=+601.181257303" watchObservedRunningTime="2026-07-09 17:49:45.09729861 +0000 UTC m=+601.183422075" Jul 09 17:49:45.119311 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:45.119257 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podStartSLOduration=2.499765204 podStartE2EDuration="34.119244517s" totalImagesPullingTime="28.168162305s" totalInitContainerRuntime="3.451317008s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:49:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:49:45.117146989 +0000 UTC m=+601.203270480" watchObservedRunningTime="2026-07-09 17:49:45.119244517 +0000 UTC m=+601.205367983" Jul 09 17:49:46.048586 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:46.048545 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Jul 09 17:49:46.049018 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:46.048593 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Jul 09 17:49:46.049018 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:46.048545 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 09 17:49:51.052883 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:51.052848 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:49:51.053348 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:51.053035 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:49:51.053348 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:51.053148 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:49:51.053348 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:51.053193 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 09 17:49:51.053795 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:51.053768 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Jul 09 17:49:51.053962 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:49:51.053943 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Jul 09 17:50:01.053323 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:01.053284 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 09 17:50:01.053763 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:01.053736 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Jul 09 17:50:01.053890 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:01.053871 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Jul 09 17:50:11.053830 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:11.053741 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Jul 09 17:50:11.054252 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:11.053739 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 09 17:50:11.054252 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:11.053863 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Jul 09 17:50:21.053244 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:21.053206 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 09 17:50:21.053711 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:21.053682 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Jul 09 17:50:21.053873 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:21.053854 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Jul 09 17:50:31.053262 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:31.053215 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 09 17:50:31.054840 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:31.053726 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Jul 09 17:50:31.054840 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:31.053859 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Jul 09 17:50:41.053481 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:41.053441 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 09 17:50:41.053920 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:41.053704 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Jul 09 17:50:41.053920 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:41.053831 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Jul 09 17:50:51.053328 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:51.053290 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 09 17:50:51.053712 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:51.053671 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Jul 09 17:50:51.055393 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:50:51.055378 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:51:01.053925 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:01.053892 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:51:01.054560 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:01.054540 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:51:41.095809 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.095731 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf"] Jul 09 17:51:41.096297 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.095983 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" containerID="cri-o://a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760" gracePeriod=30 Jul 09 17:51:41.096297 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.096027 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kube-rbac-proxy" containerID="cri-o://62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1" gracePeriod=30 Jul 09 17:51:41.152346 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.152315 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27"] Jul 09 17:51:41.152562 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.152537 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" containerID="cri-o://70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17" gracePeriod=30 Jul 09 17:51:41.152620 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.152569 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kube-rbac-proxy" containerID="cri-o://882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c" gracePeriod=30 Jul 09 17:51:41.281871 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.281838 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf"] Jul 09 17:51:41.282183 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.282154 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" containerID="cri-o://224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5" gracePeriod=30 Jul 09 17:51:41.282319 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.282277 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kube-rbac-proxy" containerID="cri-o://929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f" gracePeriod=30 Jul 09 17:51:41.532836 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.532802 2571 generic.go:356] "Generic (PLEG): container finished" podID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerID="929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f" exitCode=2 Jul 09 17:51:41.533006 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.532876 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" event={"ID":"eab88989-e0f2-4690-857a-e9bb6e6d93c5","Type":"ContainerDied","Data":"929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f"} Jul 09 17:51:41.534989 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.534966 2571 generic.go:356] "Generic (PLEG): container finished" podID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerID="62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1" exitCode=2 Jul 09 17:51:41.535115 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.535038 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" event={"ID":"423e843b-dfbe-4457-9ed1-6d9e3deec950","Type":"ContainerDied","Data":"62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1"} Jul 09 17:51:41.537063 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.537043 2571 generic.go:356] "Generic (PLEG): container finished" podID="01e3411a-227b-4060-9de4-184750ea161d" containerID="882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c" exitCode=2 Jul 09 17:51:41.537181 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:41.537070 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" event={"ID":"01e3411a-227b-4060-9de4-184750ea161d","Type":"ContainerDied","Data":"882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c"} Jul 09 17:51:44.841800 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.841770 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:51:44.983303 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.983194 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbpkq\" (UniqueName: \"kubernetes.io/projected/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-api-access-sbpkq\") pod \"423e843b-dfbe-4457-9ed1-6d9e3deec950\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " Jul 09 17:51:44.983303 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.983256 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/423e843b-dfbe-4457-9ed1-6d9e3deec950-proxy-tls\") pod \"423e843b-dfbe-4457-9ed1-6d9e3deec950\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " Jul 09 17:51:44.983520 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.983361 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-cabundle-cert\") pod \"423e843b-dfbe-4457-9ed1-6d9e3deec950\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " Jul 09 17:51:44.983520 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.983396 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/423e843b-dfbe-4457-9ed1-6d9e3deec950-kserve-provision-location\") pod \"423e843b-dfbe-4457-9ed1-6d9e3deec950\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " Jul 09 17:51:44.983520 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.983433 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-rbac-proxy-sar-config\") pod \"423e843b-dfbe-4457-9ed1-6d9e3deec950\" (UID: \"423e843b-dfbe-4457-9ed1-6d9e3deec950\") " Jul 09 17:51:44.983757 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.983728 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423e843b-dfbe-4457-9ed1-6d9e3deec950-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "423e843b-dfbe-4457-9ed1-6d9e3deec950" (UID: "423e843b-dfbe-4457-9ed1-6d9e3deec950"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:51:44.983808 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.983740 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "423e843b-dfbe-4457-9ed1-6d9e3deec950" (UID: "423e843b-dfbe-4457-9ed1-6d9e3deec950"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:51:44.983808 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.983790 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "423e843b-dfbe-4457-9ed1-6d9e3deec950" (UID: "423e843b-dfbe-4457-9ed1-6d9e3deec950"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:51:44.985541 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.985518 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-api-access-sbpkq" (OuterVolumeSpecName: "kube-api-access-sbpkq") pod "423e843b-dfbe-4457-9ed1-6d9e3deec950" (UID: "423e843b-dfbe-4457-9ed1-6d9e3deec950"). InnerVolumeSpecName "kube-api-access-sbpkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:51:44.985639 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:44.985591 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/423e843b-dfbe-4457-9ed1-6d9e3deec950-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "423e843b-dfbe-4457-9ed1-6d9e3deec950" (UID: "423e843b-dfbe-4457-9ed1-6d9e3deec950"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:51:45.084876 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.084845 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbpkq\" (UniqueName: \"kubernetes.io/projected/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-api-access-sbpkq\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:45.084876 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.084872 2571 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/423e843b-dfbe-4457-9ed1-6d9e3deec950-proxy-tls\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:45.085045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.084888 2571 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-cabundle-cert\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:45.085045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.084897 2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/423e843b-dfbe-4457-9ed1-6d9e3deec950-kserve-provision-location\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:45.085045 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.084906 2571 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/423e843b-dfbe-4457-9ed1-6d9e3deec950-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:45.557757 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.557723 2571 generic.go:356] "Generic (PLEG): container finished" podID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerID="a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760" exitCode=0 Jul 09 17:51:45.557925 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.557777 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" event={"ID":"423e843b-dfbe-4457-9ed1-6d9e3deec950","Type":"ContainerDied","Data":"a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760"} Jul 09 17:51:45.557925 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.557820 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" event={"ID":"423e843b-dfbe-4457-9ed1-6d9e3deec950","Type":"ContainerDied","Data":"bb843531870e20bcc13b4210b69081ac45c66dd3168aeba9c86a0ecb9d972d4e"} Jul 09 17:51:45.557925 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.557780 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf" Jul 09 17:51:45.557925 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.557847 2571 scope.go:117] "RemoveContainer" containerID="62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1" Jul 09 17:51:45.566968 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.566949 2571 scope.go:117] "RemoveContainer" containerID="a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760" Jul 09 17:51:45.574113 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.574095 2571 scope.go:117] "RemoveContainer" containerID="e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815" Jul 09 17:51:45.581807 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.581783 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf"] Jul 09 17:51:45.582210 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.582197 2571 scope.go:117] "RemoveContainer" containerID="62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1" Jul 09 17:51:45.582463 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:51:45.582442 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1\": container with ID starting with 62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1 not found: ID does not exist" containerID="62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1" Jul 09 17:51:45.582518 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.582471 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1"} err="failed to get container status \"62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1\": rpc error: code = NotFound desc = could not find container \"62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1\": container with ID starting with 62a5af4bddd5b59df99ecb2bb5f974e597c82f1b05141dae64b12db1325a02a1 not found: ID does not exist" Jul 09 17:51:45.582518 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.582487 2571 scope.go:117] "RemoveContainer" containerID="a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760" Jul 09 17:51:45.582704 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:51:45.582691 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760\": container with ID starting with a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760 not found: ID does not exist" containerID="a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760" Jul 09 17:51:45.582748 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.582708 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760"} err="failed to get container status \"a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760\": rpc error: code = NotFound desc = could not find container \"a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760\": container with ID starting with a2ebcbd7c363335eb0249ca7b7a9c78fce81c28e08b4d32cc3bf9b35447c1760 not found: ID does not exist" Jul 09 17:51:45.582748 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.582720 2571 scope.go:117] "RemoveContainer" containerID="e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815" Jul 09 17:51:45.582923 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:51:45.582901 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815\": container with ID starting with e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815 not found: ID does not exist" containerID="e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815" Jul 09 17:51:45.582986 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.582925 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815"} err="failed to get container status \"e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815\": rpc error: code = NotFound desc = could not find container \"e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815\": container with ID starting with e9da2eeff2dbbc992f4f770a7301b0c8c4ee9a6a37eeebda9f8e5ecc2a5db815 not found: ID does not exist" Jul 09 17:51:45.585453 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:45.585434 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2nzjf"] Jul 09 17:51:46.048889 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.048847 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused" Jul 09 17:51:46.125369 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.125339 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:51:46.237422 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.237391 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:51:46.295828 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.295796 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-kube-rbac-proxy-sar-config\") pod \"01e3411a-227b-4060-9de4-184750ea161d\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " Jul 09 17:51:46.295968 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.295838 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-cabundle-cert\") pod \"01e3411a-227b-4060-9de4-184750ea161d\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " Jul 09 17:51:46.295968 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.295877 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01e3411a-227b-4060-9de4-184750ea161d-kserve-provision-location\") pod \"01e3411a-227b-4060-9de4-184750ea161d\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " Jul 09 17:51:46.295968 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.295914 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vg8g\" (UniqueName: \"kubernetes.io/projected/01e3411a-227b-4060-9de4-184750ea161d-kube-api-access-7vg8g\") pod \"01e3411a-227b-4060-9de4-184750ea161d\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " Jul 09 17:51:46.296132 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.296060 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01e3411a-227b-4060-9de4-184750ea161d-proxy-tls\") pod \"01e3411a-227b-4060-9de4-184750ea161d\" (UID: \"01e3411a-227b-4060-9de4-184750ea161d\") " Jul 09 17:51:46.296220 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.296199 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "01e3411a-227b-4060-9de4-184750ea161d" (UID: "01e3411a-227b-4060-9de4-184750ea161d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:51:46.296313 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.296236 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "01e3411a-227b-4060-9de4-184750ea161d" (UID: "01e3411a-227b-4060-9de4-184750ea161d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:51:46.296313 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.296245 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01e3411a-227b-4060-9de4-184750ea161d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "01e3411a-227b-4060-9de4-184750ea161d" (UID: "01e3411a-227b-4060-9de4-184750ea161d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:51:46.296401 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.296383 2571 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:46.296436 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.296404 2571 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01e3411a-227b-4060-9de4-184750ea161d-cabundle-cert\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:46.296436 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.296414 2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01e3411a-227b-4060-9de4-184750ea161d-kserve-provision-location\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:46.297993 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.297972 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01e3411a-227b-4060-9de4-184750ea161d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "01e3411a-227b-4060-9de4-184750ea161d" (UID: "01e3411a-227b-4060-9de4-184750ea161d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:51:46.298066 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.297994 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01e3411a-227b-4060-9de4-184750ea161d-kube-api-access-7vg8g" (OuterVolumeSpecName: "kube-api-access-7vg8g") pod "01e3411a-227b-4060-9de4-184750ea161d" (UID: "01e3411a-227b-4060-9de4-184750ea161d"). InnerVolumeSpecName "kube-api-access-7vg8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:51:46.396828 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.396798 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjpth\" (UniqueName: \"kubernetes.io/projected/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-api-access-mjpth\") pod \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " Jul 09 17:51:46.396967 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.396848 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-rbac-proxy-sar-config\") pod \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " Jul 09 17:51:46.396967 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.396874 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-cabundle-cert\") pod \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " Jul 09 17:51:46.396967 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.396905 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eab88989-e0f2-4690-857a-e9bb6e6d93c5-proxy-tls\") pod \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " Jul 09 17:51:46.396967 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.396958 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kserve-provision-location\") pod \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\" (UID: \"eab88989-e0f2-4690-857a-e9bb6e6d93c5\") " Jul 09 17:51:46.397175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.397154 2571 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01e3411a-227b-4060-9de4-184750ea161d-proxy-tls\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:46.397175 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.397179 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vg8g\" (UniqueName: \"kubernetes.io/projected/01e3411a-227b-4060-9de4-184750ea161d-kube-api-access-7vg8g\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:46.397322 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.397255 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "eab88989-e0f2-4690-857a-e9bb6e6d93c5" (UID: "eab88989-e0f2-4690-857a-e9bb6e6d93c5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:51:46.397322 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.397301 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "eab88989-e0f2-4690-857a-e9bb6e6d93c5" (UID: "eab88989-e0f2-4690-857a-e9bb6e6d93c5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:51:46.397408 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.397330 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "eab88989-e0f2-4690-857a-e9bb6e6d93c5" (UID: "eab88989-e0f2-4690-857a-e9bb6e6d93c5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:51:46.398965 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.398939 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-api-access-mjpth" (OuterVolumeSpecName: "kube-api-access-mjpth") pod "eab88989-e0f2-4690-857a-e9bb6e6d93c5" (UID: "eab88989-e0f2-4690-857a-e9bb6e6d93c5"). InnerVolumeSpecName "kube-api-access-mjpth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:51:46.399057 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.398966 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eab88989-e0f2-4690-857a-e9bb6e6d93c5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "eab88989-e0f2-4690-857a-e9bb6e6d93c5" (UID: "eab88989-e0f2-4690-857a-e9bb6e6d93c5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:51:46.482604 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.477774 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" path="/var/lib/kubelet/pods/423e843b-dfbe-4457-9ed1-6d9e3deec950/volumes" Jul 09 17:51:46.497497 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.497475 2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kserve-provision-location\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:46.497602 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.497499 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjpth\" (UniqueName: \"kubernetes.io/projected/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-api-access-mjpth\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:46.497602 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.497513 2571 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:46.497602 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.497550 2571 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/eab88989-e0f2-4690-857a-e9bb6e6d93c5-cabundle-cert\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:46.497602 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.497565 2571 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eab88989-e0f2-4690-857a-e9bb6e6d93c5-proxy-tls\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 17:51:46.563865 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.563829 2571 generic.go:356] "Generic (PLEG): container finished" podID="01e3411a-227b-4060-9de4-184750ea161d" containerID="70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17" exitCode=0 Jul 09 17:51:46.564025 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.563901 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" Jul 09 17:51:46.564025 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.563897 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" event={"ID":"01e3411a-227b-4060-9de4-184750ea161d","Type":"ContainerDied","Data":"70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17"} Jul 09 17:51:46.564155 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.564084 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" event={"ID":"01e3411a-227b-4060-9de4-184750ea161d","Type":"ContainerDied","Data":"2b53a4ad722d03647daedce86b2657825e8ce4bfb5472a6f1eeac82749385d8c"} Jul 09 17:51:46.564155 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.564144 2571 scope.go:117] "RemoveContainer" containerID="882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c" Jul 09 17:51:46.566657 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.566330 2571 generic.go:356] "Generic (PLEG): container finished" podID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerID="224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5" exitCode=0 Jul 09 17:51:46.566657 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.566454 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" event={"ID":"eab88989-e0f2-4690-857a-e9bb6e6d93c5","Type":"ContainerDied","Data":"224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5"} Jul 09 17:51:46.566657 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.566506 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" event={"ID":"eab88989-e0f2-4690-857a-e9bb6e6d93c5","Type":"ContainerDied","Data":"8127819ea34dd7c8858af0d70730dbe9de2da4e98802e3414b24a929f8ad22bb"} Jul 09 17:51:46.566657 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.566617 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf" Jul 09 17:51:46.573555 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.573536 2571 scope.go:117] "RemoveContainer" containerID="70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17" Jul 09 17:51:46.580871 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.580851 2571 scope.go:117] "RemoveContainer" containerID="bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831" Jul 09 17:51:46.585018 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.584992 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27"] Jul 09 17:51:46.595001 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.594977 2571 scope.go:117] "RemoveContainer" containerID="882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c" Jul 09 17:51:46.595364 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:51:46.595339 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c\": container with ID starting with 882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c not found: ID does not exist" containerID="882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c" Jul 09 17:51:46.595517 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.595489 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c"} err="failed to get container status \"882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c\": rpc error: code = NotFound desc = could not find container \"882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c\": container with ID starting with 882d85fe9ee7b59d4ad5ba7c714f7f9716cbd5d0adda9970e173ca5155a3e81c not found: ID does not exist" Jul 09 17:51:46.595634 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.595623 2571 scope.go:117] "RemoveContainer" containerID="70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17" Jul 09 17:51:46.595989 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:51:46.595971 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17\": container with ID starting with 70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17 not found: ID does not exist" containerID="70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17" Jul 09 17:51:46.596108 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.596089 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17"} err="failed to get container status \"70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17\": rpc error: code = NotFound desc = could not find container \"70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17\": container with ID starting with 70ddf001c8aff03ec454b0bbf9284bc9d942fd1e8c542ad6320cba3f71bd5f17 not found: ID does not exist" Jul 09 17:51:46.596229 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.596185 2571 scope.go:117] "RemoveContainer" containerID="bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831" Jul 09 17:51:46.596763 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:51:46.596743 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831\": container with ID starting with bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831 not found: ID does not exist" containerID="bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831" Jul 09 17:51:46.596887 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.596868 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831"} err="failed to get container status \"bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831\": rpc error: code = NotFound desc = could not find container \"bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831\": container with ID starting with bf44764df3c0ef51a9163d6d3827839b1bd45895c003f85cf14d19e64e939831 not found: ID does not exist" Jul 09 17:51:46.596974 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.596964 2571 scope.go:117] "RemoveContainer" containerID="929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f" Jul 09 17:51:46.598526 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.598506 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27"] Jul 09 17:51:46.605078 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.605050 2571 scope.go:117] "RemoveContainer" containerID="224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5" Jul 09 17:51:46.609247 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.609216 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf"] Jul 09 17:51:46.613334 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.613314 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64f67649c4-7k6gf"] Jul 09 17:51:46.613521 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.613506 2571 scope.go:117] "RemoveContainer" containerID="bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53" Jul 09 17:51:46.620967 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.620948 2571 scope.go:117] "RemoveContainer" containerID="929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f" Jul 09 17:51:46.621207 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:51:46.621189 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f\": container with ID starting with 929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f not found: ID does not exist" containerID="929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f" Jul 09 17:51:46.621251 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.621214 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f"} err="failed to get container status \"929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f\": rpc error: code = NotFound desc = could not find container \"929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f\": container with ID starting with 929442a1b40473939ae397c5f0f70000f6931eed8b1c79c74e43418fb7e7a91f not found: ID does not exist" Jul 09 17:51:46.621251 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.621228 2571 scope.go:117] "RemoveContainer" containerID="224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5" Jul 09 17:51:46.621445 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:51:46.621430 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5\": container with ID starting with 224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5 not found: ID does not exist" containerID="224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5" Jul 09 17:51:46.621485 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.621448 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5"} err="failed to get container status \"224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5\": rpc error: code = NotFound desc = could not find container \"224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5\": container with ID starting with 224a3862b92bb07c9ebdfbbfa8de593a9525e23ecda89c702a67c5bf711a55b5 not found: ID does not exist" Jul 09 17:51:46.621485 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.621461 2571 scope.go:117] "RemoveContainer" containerID="bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53" Jul 09 17:51:46.621676 ip-10-0-133-63 kubenswrapper[2571]: E0709 17:51:46.621663 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53\": container with ID starting with bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53 not found: ID does not exist" containerID="bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53" Jul 09 17:51:46.621710 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:46.621679 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53"} err="failed to get container status \"bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53\": rpc error: code = NotFound desc = could not find container \"bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53\": container with ID starting with bd47785688490754388faaa239d1c084d9f61bcf32f912559ab4f48d9a187d53 not found: ID does not exist" Jul 09 17:51:47.049215 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:47.049173 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6d7745cf6b-7hk27" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 09 17:51:48.478234 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:48.478197 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01e3411a-227b-4060-9de4-184750ea161d" path="/var/lib/kubelet/pods/01e3411a-227b-4060-9de4-184750ea161d/volumes" Jul 09 17:51:48.478710 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:51:48.478695 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" path="/var/lib/kubelet/pods/eab88989-e0f2-4690-857a-e9bb6e6d93c5/volumes" Jul 09 17:54:44.356929 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:54:44.356899 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:54:44.357493 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:54:44.357245 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:54:44.364242 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:54:44.364220 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:54:44.364827 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:54:44.364810 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:59:44.383295 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:59:44.383253 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:59:44.385389 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:59:44.385019 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 17:59:44.390332 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:59:44.390314 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 17:59:44.392098 ip-10-0-133-63 kubenswrapper[2571]: I0709 17:59:44.392080 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:04:44.408785 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:04:44.408665 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:04:44.411861 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:04:44.410752 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:04:44.416409 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:04:44.416390 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:04:44.417992 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:04:44.417975 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:09:44.434682 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:09:44.434573 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:09:44.441315 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:09:44.436904 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:09:44.441957 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:09:44.441939 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:09:44.444072 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:09:44.444057 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:14:44.462388 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:14:44.462251 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:14:44.465939 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:14:44.465918 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:14:44.469668 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:14:44.469647 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:14:44.473735 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:14:44.473717 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:19:44.489513 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:19:44.489408 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:19:44.494032 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:19:44.494010 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:19:44.496950 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:19:44.496932 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:19:44.501154 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:19:44.501138 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:24:44.515122 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:24:44.515007 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:24:44.520985 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:24:44.520963 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:24:44.521933 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:24:44.521910 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:24:44.527741 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:24:44.527724 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:28:52.101294 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101249 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qvckn/must-gather-z5j48"] Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101601 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="storage-initializer" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101615 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="storage-initializer" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101625 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101633 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101645 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kube-rbac-proxy" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101651 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kube-rbac-proxy" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101661 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kube-rbac-proxy" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101670 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kube-rbac-proxy" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101682 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="storage-initializer" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101687 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="storage-initializer" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101694 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="storage-initializer" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101700 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="storage-initializer" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101707 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101712 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101719 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kube-rbac-proxy" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101723 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kube-rbac-proxy" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101735 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" Jul 09 18:28:52.101756 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101744 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" Jul 09 18:28:52.102295 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101792 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kserve-container" Jul 09 18:28:52.102295 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101801 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kserve-container" Jul 09 18:28:52.102295 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101807 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kserve-container" Jul 09 18:28:52.102295 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101814 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="01e3411a-227b-4060-9de4-184750ea161d" containerName="kube-rbac-proxy" Jul 09 18:28:52.102295 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101823 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="eab88989-e0f2-4690-857a-e9bb6e6d93c5" containerName="kube-rbac-proxy" Jul 09 18:28:52.102295 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.101833 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="423e843b-dfbe-4457-9ed1-6d9e3deec950" containerName="kube-rbac-proxy" Jul 09 18:28:52.104669 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.104653 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:28:52.107153 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.107127 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qvckn\"/\"default-dockercfg-vpjgb\"" Jul 09 18:28:52.107521 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.107503 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qvckn\"/\"kube-root-ca.crt\"" Jul 09 18:28:52.107616 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.107542 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qvckn\"/\"openshift-service-ca.crt\"" Jul 09 18:28:52.113702 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.113681 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qvckn/must-gather-z5j48"] Jul 09 18:28:52.204658 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.204627 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzkrt\" (UniqueName: \"kubernetes.io/projected/ba87c6b5-894a-473b-bf44-3cf9c50a7144-kube-api-access-mzkrt\") pod \"must-gather-z5j48\" (UID: \"ba87c6b5-894a-473b-bf44-3cf9c50a7144\") " pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:28:52.204819 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.204678 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ba87c6b5-894a-473b-bf44-3cf9c50a7144-must-gather-output\") pod \"must-gather-z5j48\" (UID: \"ba87c6b5-894a-473b-bf44-3cf9c50a7144\") " pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:28:52.305436 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.305402 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzkrt\" (UniqueName: \"kubernetes.io/projected/ba87c6b5-894a-473b-bf44-3cf9c50a7144-kube-api-access-mzkrt\") pod \"must-gather-z5j48\" (UID: \"ba87c6b5-894a-473b-bf44-3cf9c50a7144\") " pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:28:52.305589 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.305463 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ba87c6b5-894a-473b-bf44-3cf9c50a7144-must-gather-output\") pod \"must-gather-z5j48\" (UID: \"ba87c6b5-894a-473b-bf44-3cf9c50a7144\") " pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:28:52.305766 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.305746 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ba87c6b5-894a-473b-bf44-3cf9c50a7144-must-gather-output\") pod \"must-gather-z5j48\" (UID: \"ba87c6b5-894a-473b-bf44-3cf9c50a7144\") " pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:28:52.314496 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.314476 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzkrt\" (UniqueName: \"kubernetes.io/projected/ba87c6b5-894a-473b-bf44-3cf9c50a7144-kube-api-access-mzkrt\") pod \"must-gather-z5j48\" (UID: \"ba87c6b5-894a-473b-bf44-3cf9c50a7144\") " pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:28:52.414710 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.414647 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:28:52.540007 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.539919 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qvckn/must-gather-z5j48"] Jul 09 18:28:52.542212 ip-10-0-133-63 kubenswrapper[2571]: W0709 18:28:52.542184 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba87c6b5_894a_473b_bf44_3cf9c50a7144.slice/crio-7ab471a975fe1c458022f534dc01acdb6dcee267fab6976837e3098445ad8061 WatchSource:0}: Error finding container 7ab471a975fe1c458022f534dc01acdb6dcee267fab6976837e3098445ad8061: Status 404 returned error can't find the container with id 7ab471a975fe1c458022f534dc01acdb6dcee267fab6976837e3098445ad8061 Jul 09 18:28:52.543977 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:52.543962 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:28:53.197713 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:53.197646 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qvckn/must-gather-z5j48" event={"ID":"ba87c6b5-894a-473b-bf44-3cf9c50a7144","Type":"ContainerStarted","Data":"7ab471a975fe1c458022f534dc01acdb6dcee267fab6976837e3098445ad8061"} Jul 09 18:28:59.234107 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:59.234069 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qvckn/must-gather-z5j48" event={"ID":"ba87c6b5-894a-473b-bf44-3cf9c50a7144","Type":"ContainerStarted","Data":"ecff5cffe241a986723819e5b6ca97220c6ed78da08d33802a6986a244fad7ca"} Jul 09 18:28:59.234532 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:59.234116 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qvckn/must-gather-z5j48" event={"ID":"ba87c6b5-894a-473b-bf44-3cf9c50a7144","Type":"ContainerStarted","Data":"035fcb9acda10a7689a7ffe1cb191b7bd3c80761cb66e0dd3498883ecfd32e2b"} Jul 09 18:28:59.258126 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:28:59.258076 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qvckn/must-gather-z5j48" podStartSLOduration=1.3430028059999999 podStartE2EDuration="7.258060851s" totalImagesPullingTime="5.915058045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:28:59.257218552 +0000 UTC m=+2955.343342019" watchObservedRunningTime="2026-07-09 18:28:59.258060851 +0000 UTC m=+2955.344184316" Jul 09 18:29:22.334602 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:22.334568 2571 generic.go:356] "Generic (PLEG): container finished" podID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" containerID="035fcb9acda10a7689a7ffe1cb191b7bd3c80761cb66e0dd3498883ecfd32e2b" exitCode=0 Jul 09 18:29:22.335025 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:22.334614 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qvckn/must-gather-z5j48" event={"ID":"ba87c6b5-894a-473b-bf44-3cf9c50a7144","Type":"ContainerDied","Data":"035fcb9acda10a7689a7ffe1cb191b7bd3c80761cb66e0dd3498883ecfd32e2b"} Jul 09 18:29:22.335025 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:22.334889 2571 scope.go:117] "RemoveContainer" containerID="035fcb9acda10a7689a7ffe1cb191b7bd3c80761cb66e0dd3498883ecfd32e2b" Jul 09 18:29:22.768238 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:22.768208 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-qvckn_must-gather-z5j48_ba87c6b5-894a-473b-bf44-3cf9c50a7144/gather/0.log" Jul 09 18:29:25.869676 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:25.869643 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pdswh_5890960e-4225-4cae-83d2-a5749bb9d755/global-pull-secret-syncer/0.log" Jul 09 18:29:26.034677 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:26.034647 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rcl6f_d7a5d6b3-195b-4cbf-a5d6-7c7de44f3d51/konnectivity-agent/0.log" Jul 09 18:29:26.105345 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:26.105319 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-63.ec2.internal_1a96eeef141b89fa6a1cb2d723efd4cb/haproxy/0.log" Jul 09 18:29:28.216079 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.216048 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-qvckn/must-gather-z5j48"] Jul 09 18:29:28.216569 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.216235 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-qvckn/must-gather-z5j48" podUID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" containerName="copy" containerID="cri-o://ecff5cffe241a986723819e5b6ca97220c6ed78da08d33802a6986a244fad7ca" gracePeriod=2 Jul 09 18:29:28.226754 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.226731 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-qvckn/must-gather-z5j48"] Jul 09 18:29:28.228222 ip-10-0-133-63 kubenswrapper[2571]: E0709 18:29:28.228026 2571 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ba87c6b5-894a-473b-bf44-3cf9c50a7144\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-must-gather-qvckn\"/\"must-gather-z5j48\": pods \"must-gather-z5j48\" not found" pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:29:28.360674 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.360626 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-qvckn_must-gather-z5j48_ba87c6b5-894a-473b-bf44-3cf9c50a7144/copy/0.log" Jul 09 18:29:28.361186 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.361057 2571 generic.go:356] "Generic (PLEG): container finished" podID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" containerID="ecff5cffe241a986723819e5b6ca97220c6ed78da08d33802a6986a244fad7ca" exitCode=143 Jul 09 18:29:28.446336 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.446315 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-qvckn_must-gather-z5j48_ba87c6b5-894a-473b-bf44-3cf9c50a7144/copy/0.log" Jul 09 18:29:28.448939 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.448922 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:29:28.451949 ip-10-0-133-63 kubenswrapper[2571]: E0709 18:29:28.451925 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-z5j48\" is forbidden: User \"system:node:ip-10-0-133-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-qvckn\": no relationship found between node 'ip-10-0-133-63.ec2.internal' and this object" podUID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:29:28.534397 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.534346 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ba87c6b5-894a-473b-bf44-3cf9c50a7144-must-gather-output\") pod \"ba87c6b5-894a-473b-bf44-3cf9c50a7144\" (UID: \"ba87c6b5-894a-473b-bf44-3cf9c50a7144\") " Jul 09 18:29:28.534502 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.534415 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzkrt\" (UniqueName: \"kubernetes.io/projected/ba87c6b5-894a-473b-bf44-3cf9c50a7144-kube-api-access-mzkrt\") pod \"ba87c6b5-894a-473b-bf44-3cf9c50a7144\" (UID: \"ba87c6b5-894a-473b-bf44-3cf9c50a7144\") " Jul 09 18:29:28.535637 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.535610 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba87c6b5-894a-473b-bf44-3cf9c50a7144-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "ba87c6b5-894a-473b-bf44-3cf9c50a7144" (UID: "ba87c6b5-894a-473b-bf44-3cf9c50a7144"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:29:28.536605 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.536582 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba87c6b5-894a-473b-bf44-3cf9c50a7144-kube-api-access-mzkrt" (OuterVolumeSpecName: "kube-api-access-mzkrt") pod "ba87c6b5-894a-473b-bf44-3cf9c50a7144" (UID: "ba87c6b5-894a-473b-bf44-3cf9c50a7144"). InnerVolumeSpecName "kube-api-access-mzkrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:29:28.635101 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.635078 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzkrt\" (UniqueName: \"kubernetes.io/projected/ba87c6b5-894a-473b-bf44-3cf9c50a7144-kube-api-access-mzkrt\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 18:29:28.635101 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:28.635099 2571 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ba87c6b5-894a-473b-bf44-3cf9c50a7144-must-gather-output\") on node \"ip-10-0-133-63.ec2.internal\" DevicePath \"\"" Jul 09 18:29:29.366131 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:29.366102 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-qvckn_must-gather-z5j48_ba87c6b5-894a-473b-bf44-3cf9c50a7144/copy/0.log" Jul 09 18:29:29.366552 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:29.366510 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qvckn/must-gather-z5j48" Jul 09 18:29:29.367404 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:29.366495 2571 scope.go:117] "RemoveContainer" containerID="ecff5cffe241a986723819e5b6ca97220c6ed78da08d33802a6986a244fad7ca" Jul 09 18:29:29.380789 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:29.380770 2571 scope.go:117] "RemoveContainer" containerID="035fcb9acda10a7689a7ffe1cb191b7bd3c80761cb66e0dd3498883ecfd32e2b" Jul 09 18:29:29.880831 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:29.880810 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28af0edb-e52f-4f2f-8413-028d03f56308/alertmanager/0.log" Jul 09 18:29:29.921338 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:29.921311 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28af0edb-e52f-4f2f-8413-028d03f56308/config-reloader/0.log" Jul 09 18:29:29.961091 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:29.961070 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28af0edb-e52f-4f2f-8413-028d03f56308/kube-rbac-proxy-web/0.log" Jul 09 18:29:29.995759 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:29.995742 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28af0edb-e52f-4f2f-8413-028d03f56308/kube-rbac-proxy/0.log" Jul 09 18:29:30.022144 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.022123 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28af0edb-e52f-4f2f-8413-028d03f56308/kube-rbac-proxy-metric/0.log" Jul 09 18:29:30.057312 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.057297 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28af0edb-e52f-4f2f-8413-028d03f56308/prom-label-proxy/0.log" Jul 09 18:29:30.086451 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.086433 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28af0edb-e52f-4f2f-8413-028d03f56308/init-config-reloader/0.log" Jul 09 18:29:30.132826 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.132774 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-z75dj_5e90ec89-3529-45c0-8422-fe0c83972cd4/cluster-monitoring-operator/0.log" Jul 09 18:29:30.167305 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.167288 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-n7kl8_7b9fe8bd-8392-422a-a289-f4d2a27dfc40/kube-state-metrics/0.log" Jul 09 18:29:30.196561 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.196523 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-n7kl8_7b9fe8bd-8392-422a-a289-f4d2a27dfc40/kube-rbac-proxy-main/0.log" Jul 09 18:29:30.223641 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.223619 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-n7kl8_7b9fe8bd-8392-422a-a289-f4d2a27dfc40/kube-rbac-proxy-self/0.log" Jul 09 18:29:30.303801 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.303784 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-zvsfx_0a4844e3-3459-4990-838c-b403938b04a3/monitoring-plugin/0.log" Jul 09 18:29:30.476827 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.476749 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" path="/var/lib/kubelet/pods/ba87c6b5-894a-473b-bf44-3cf9c50a7144/volumes" Jul 09 18:29:30.530863 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.530839 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xplkk_9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5/node-exporter/0.log" Jul 09 18:29:30.555207 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.555189 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xplkk_9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5/kube-rbac-proxy/0.log" Jul 09 18:29:30.583037 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.583017 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xplkk_9cfeeab1-ceb4-4922-81dc-3ebf5be9c8f5/init-textfile/0.log" Jul 09 18:29:30.725391 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.725354 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_460c63ca-2179-47a8-b97a-8c88bf73c7d3/prometheus/0.log" Jul 09 18:29:30.747899 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.747850 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_460c63ca-2179-47a8-b97a-8c88bf73c7d3/config-reloader/0.log" Jul 09 18:29:30.770959 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.770939 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_460c63ca-2179-47a8-b97a-8c88bf73c7d3/thanos-sidecar/0.log" Jul 09 18:29:30.794999 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.794980 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_460c63ca-2179-47a8-b97a-8c88bf73c7d3/kube-rbac-proxy-web/0.log" Jul 09 18:29:30.817701 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.817686 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_460c63ca-2179-47a8-b97a-8c88bf73c7d3/kube-rbac-proxy/0.log" Jul 09 18:29:30.846555 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.846533 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_460c63ca-2179-47a8-b97a-8c88bf73c7d3/kube-rbac-proxy-thanos/0.log" Jul 09 18:29:30.873757 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.873735 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_460c63ca-2179-47a8-b97a-8c88bf73c7d3/init-config-reloader/0.log" Jul 09 18:29:30.905822 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.905801 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-sqwxw_58e45f2e-a384-481f-b163-fa4accf8c0a1/prometheus-operator/0.log" Jul 09 18:29:30.932656 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.932636 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-sqwxw_58e45f2e-a384-481f-b163-fa4accf8c0a1/kube-rbac-proxy/0.log" Jul 09 18:29:30.960673 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:30.960652 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-6l54l_813a8dc1-6c10-444d-8dda-69d2c32c9acc/prometheus-operator-admission-webhook/0.log" Jul 09 18:29:31.001087 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:31.001031 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f759875c-np9h5_f1b32223-ef89-4e8a-84eb-0c9b51ea2213/telemeter-client/0.log" Jul 09 18:29:31.029193 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:31.029176 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f759875c-np9h5_f1b32223-ef89-4e8a-84eb-0c9b51ea2213/reload/0.log" Jul 09 18:29:31.057099 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:31.057081 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f759875c-np9h5_f1b32223-ef89-4e8a-84eb-0c9b51ea2213/kube-rbac-proxy/0.log" Jul 09 18:29:33.081253 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:33.081222 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:29:33.085046 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:33.085026 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/2.log" Jul 09 18:29:34.140698 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.140670 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-fkrwx_75bd3409-8739-41bc-88eb-501f310b7ecb/volume-data-source-validator/0.log" Jul 09 18:29:34.207169 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.207139 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7"] Jul 09 18:29:34.207465 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.207452 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" containerName="copy" Jul 09 18:29:34.207465 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.207466 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" containerName="copy" Jul 09 18:29:34.207555 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.207482 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" containerName="gather" Jul 09 18:29:34.207555 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.207488 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" containerName="gather" Jul 09 18:29:34.207555 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.207537 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" containerName="copy" Jul 09 18:29:34.207555 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.207549 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba87c6b5-894a-473b-bf44-3cf9c50a7144" containerName="gather" Jul 09 18:29:34.212430 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.212408 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.215007 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.214987 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q7djf\"/\"openshift-service-ca.crt\"" Jul 09 18:29:34.215080 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.215012 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q7djf\"/\"default-dockercfg-n8q72\"" Jul 09 18:29:34.216039 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.216023 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q7djf\"/\"kube-root-ca.crt\"" Jul 09 18:29:34.220490 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.220464 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7"] Jul 09 18:29:34.276886 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.276863 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-proc\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.277005 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.276905 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctbx4\" (UniqueName: \"kubernetes.io/projected/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-kube-api-access-ctbx4\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.277005 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.276934 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-podres\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.277005 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.276959 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-lib-modules\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.277005 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.276994 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-sys\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.378369 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.378334 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-sys\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.378369 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.378377 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-proc\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.378515 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.378447 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-proc\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.378515 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.378451 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-sys\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.378515 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.378471 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctbx4\" (UniqueName: \"kubernetes.io/projected/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-kube-api-access-ctbx4\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.378515 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.378499 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-podres\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.378641 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.378523 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-lib-modules\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.378641 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.378620 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-lib-modules\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.378641 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.378625 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-podres\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.387726 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.387702 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctbx4\" (UniqueName: \"kubernetes.io/projected/d8efaef7-6f11-43d6-bd22-25bafb1d9b69-kube-api-access-ctbx4\") pod \"perf-node-gather-daemonset-ldfm7\" (UID: \"d8efaef7-6f11-43d6-bd22-25bafb1d9b69\") " pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.522541 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.522476 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:34.649307 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:34.649246 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7"] Jul 09 18:29:34.651492 ip-10-0-133-63 kubenswrapper[2571]: W0709 18:29:34.651456 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd8efaef7_6f11_43d6_bd22_25bafb1d9b69.slice/crio-9e4ee597a8de4aa857dd6e3157353107826ea27636fd5e8870d52726b1b7e117 WatchSource:0}: Error finding container 9e4ee597a8de4aa857dd6e3157353107826ea27636fd5e8870d52726b1b7e117: Status 404 returned error can't find the container with id 9e4ee597a8de4aa857dd6e3157353107826ea27636fd5e8870d52726b1b7e117 Jul 09 18:29:35.201293 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:35.201247 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7sfwq_2bc75a7a-e966-43a2-a3e3-f946bd253e69/dns/0.log" Jul 09 18:29:35.224954 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:35.224929 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7sfwq_2bc75a7a-e966-43a2-a3e3-f946bd253e69/kube-rbac-proxy/0.log" Jul 09 18:29:35.306727 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:35.306707 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nzt4v_f7e2b98f-c7e7-4796-b08e-820999127101/dns-node-resolver/0.log" Jul 09 18:29:35.392735 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:35.392705 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" event={"ID":"d8efaef7-6f11-43d6-bd22-25bafb1d9b69","Type":"ContainerStarted","Data":"06c6f5b498a33c0e1233774596c23e343fddfbc06b55583d544d551231305e53"} Jul 09 18:29:35.392873 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:35.392755 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" event={"ID":"d8efaef7-6f11-43d6-bd22-25bafb1d9b69","Type":"ContainerStarted","Data":"9e4ee597a8de4aa857dd6e3157353107826ea27636fd5e8870d52726b1b7e117"} Jul 09 18:29:35.392873 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:35.392805 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:35.415068 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:35.415018 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" podStartSLOduration=1.415003247 podStartE2EDuration="1.415003247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:29:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:29:35.414508317 +0000 UTC m=+2991.500631804" watchObservedRunningTime="2026-07-09 18:29:35.415003247 +0000 UTC m=+2991.501126712" Jul 09 18:29:35.890177 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:35.890149 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-czxd6_1654e8a4-0a47-4d4f-a3de-35a68c06bcea/node-ca/0.log" Jul 09 18:29:37.108006 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:37.107976 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lggb2_39bc423c-6220-41f9-a806-3208df828cbf/serve-healthcheck-canary/0.log" Jul 09 18:29:37.599044 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:37.598998 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qxfks_193ce01b-0b42-4397-9168-2848c5d8fb92/kube-rbac-proxy/0.log" Jul 09 18:29:37.619694 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:37.619673 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qxfks_193ce01b-0b42-4397-9168-2848c5d8fb92/exporter/0.log" Jul 09 18:29:37.642026 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:37.641993 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qxfks_193ce01b-0b42-4397-9168-2848c5d8fb92/extractor/0.log" Jul 09 18:29:40.016329 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:40.016298 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-2q7j4_76d8ab87-2c47-4053-a99d-dd38c95741fa/s3-init/0.log" Jul 09 18:29:40.033625 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:40.033600 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-2q7j4_76d8ab87-2c47-4053-a99d-dd38c95741fa/download-hf-model/0.log" Jul 09 18:29:41.406188 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:41.406163 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q7djf/perf-node-gather-daemonset-ldfm7" Jul 09 18:29:44.540775 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:44.540675 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:29:44.563522 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:44.548848 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d79td_04a9b6e5-8ddf-4e6f-ad75-e746722fcb12/console-operator/1.log" Jul 09 18:29:44.563522 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:44.548955 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:29:44.563522 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:44.555767 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:29:45.784133 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:45.784062 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-px9wm_1b0c760e-92c9-4e00-bf4e-bf578db74755/kube-multus-additional-cni-plugins/0.log" Jul 09 18:29:45.808352 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:45.808325 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-px9wm_1b0c760e-92c9-4e00-bf4e-bf578db74755/egress-router-binary-copy/0.log" Jul 09 18:29:45.830844 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:45.830821 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-px9wm_1b0c760e-92c9-4e00-bf4e-bf578db74755/cni-plugins/0.log" Jul 09 18:29:45.855028 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:45.855008 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-px9wm_1b0c760e-92c9-4e00-bf4e-bf578db74755/bond-cni-plugin/0.log" Jul 09 18:29:45.881213 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:45.881195 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-px9wm_1b0c760e-92c9-4e00-bf4e-bf578db74755/routeoverride-cni/0.log" Jul 09 18:29:45.906957 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:45.906941 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-px9wm_1b0c760e-92c9-4e00-bf4e-bf578db74755/whereabouts-cni-bincopy/0.log" Jul 09 18:29:45.930430 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:45.930411 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-px9wm_1b0c760e-92c9-4e00-bf4e-bf578db74755/whereabouts-cni/0.log" Jul 09 18:29:46.136313 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:46.136291 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fwnfj_0f515f8f-2a9d-40fc-8049-41438737d2bb/kube-multus/0.log" Jul 09 18:29:46.288785 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:46.288762 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lwssb_02523ead-082e-41e1-ab84-648a74b97bd2/network-metrics-daemon/0.log" Jul 09 18:29:46.310707 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:46.310683 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lwssb_02523ead-082e-41e1-ab84-648a74b97bd2/kube-rbac-proxy/0.log" Jul 09 18:29:47.293159 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:47.293122 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-controller/0.log" Jul 09 18:29:47.316354 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:47.316301 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/1.log" Jul 09 18:29:47.316503 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:47.316391 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovn-acl-logging/0.log" Jul 09 18:29:47.334496 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:47.334476 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/kube-rbac-proxy-node/0.log" Jul 09 18:29:47.357475 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:47.357454 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/kube-rbac-proxy-ovn-metrics/0.log" Jul 09 18:29:47.378601 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:47.378581 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/northd/0.log" Jul 09 18:29:47.402508 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:47.402489 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/nbdb/0.log" Jul 09 18:29:47.427183 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:47.427163 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/sbdb/0.log" Jul 09 18:29:47.475785 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:47.475764 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b547z_fede05a0-241f-47c0-aa3b-db1a5ccdbe63/ovnkube-controller/0.log" Jul 09 18:29:48.766692 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:48.766668 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-2p7ff_c86cda2a-444c-431e-b014-9cef96fe3d8e/network-check-target-container/0.log" Jul 09 18:29:49.696009 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:49.695984 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-fvp4m_2b67de07-eb02-48c3-9e6e-31f3b83d24b7/iptables-alerter/0.log" Jul 09 18:29:50.421151 ip-10-0-133-63 kubenswrapper[2571]: I0709 18:29:50.421123 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-mm7d7_d3fd70bf-947f-4743-a769-f926c81ebf21/tuned/0.log"