Jul 29 15:01:15.970899 ip-10-0-142-104 systemd[1]: Starting Kubernetes Kubelet... Jul 29 15:01:16.372071 ip-10-0-142-104 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 15:01:16.372071 ip-10-0-142-104 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 29 15:01:16.372071 ip-10-0-142-104 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 15:01:16.372071 ip-10-0-142-104 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 29 15:01:16.372071 ip-10-0-142-104 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 15:01:16.372899 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.372748 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 29 15:01:16.376310 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376293 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 15:01:16.376310 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376310 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376313 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376317 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376320 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376324 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376327 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376329 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376332 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376335 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376337 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376340 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376342 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376345 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376348 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376350 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376352 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376355 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376359 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376362 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376365 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 15:01:16.376382 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376367 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376370 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376372 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376375 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376377 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376380 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376382 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376385 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376388 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376390 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376393 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376397 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376401 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376404 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376407 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376409 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376412 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376415 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376417 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376420 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 15:01:16.376854 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376422 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376424 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376427 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376429 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376432 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376434 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376437 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376439 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376441 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376445 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376447 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376451 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376454 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376458 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376461 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376464 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376467 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376470 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376473 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 15:01:16.377333 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376476 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376478 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376481 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376484 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376488 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376491 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376494 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376496 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376499 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376501 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376504 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376506 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376509 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376511 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376514 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376517 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376520 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376522 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376525 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 15:01:16.377797 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376527 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376530 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376533 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376535 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376537 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376540 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376542 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376545 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376549 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376552 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376554 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376557 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376559 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376562 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376565 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376567 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376570 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376572 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376575 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.376578 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 15:01:16.378256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377278 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377284 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377287 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377292 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377295 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377298 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377301 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377304 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377307 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377310 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377313 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377315 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377318 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377320 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377323 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377325 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377328 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377330 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377333 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377335 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 15:01:16.378774 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377339 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377342 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377344 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377347 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377350 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377352 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377355 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377357 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377359 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377362 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377364 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377366 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377369 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377371 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377373 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377376 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377378 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377380 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377384 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377386 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 15:01:16.379261 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377389 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377391 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377393 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377396 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377398 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377401 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377403 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377407 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377409 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377411 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377414 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377418 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377422 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377425 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377427 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377430 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377432 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377435 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377437 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 15:01:16.379750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377440 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377442 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377445 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377447 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377449 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377452 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377454 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377457 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377459 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377462 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377464 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377466 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377469 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377472 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377475 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377477 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377479 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377482 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377484 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377486 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 15:01:16.380210 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377489 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377491 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377494 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377496 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377500 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377503 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377505 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377509 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377511 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377515 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377519 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377522 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377524 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377527 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377530 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377533 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377535 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377538 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377540 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 15:01:16.380695 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377543 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377942 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377948 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377951 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377955 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377959 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377962 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377964 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377967 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377969 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377972 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377974 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377977 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377979 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377982 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377984 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377987 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377990 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377993 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 15:01:16.381164 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377996 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.377998 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378001 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378004 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378006 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378009 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378011 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378014 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378016 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378019 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378021 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378024 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378026 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378028 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378032 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378034 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378037 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378039 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378042 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 15:01:16.381674 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378045 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378047 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378049 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378052 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378055 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378057 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378059 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378062 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378065 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378068 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378070 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378073 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378075 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378078 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378080 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378083 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378085 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378088 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378090 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378093 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 15:01:16.382127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378095 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378098 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378100 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378104 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378106 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378109 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378111 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378113 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378116 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378119 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378121 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378123 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378126 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378129 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378132 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378134 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378136 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378139 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378143 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378146 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 15:01:16.382653 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378149 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378152 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378155 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378158 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378160 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378163 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378165 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378168 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378170 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378172 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378175 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378177 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378180 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378183 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378186 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378189 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378191 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378193 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378196 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 15:01:16.383127 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378198 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378201 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378204 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378279 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378286 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378292 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378297 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378302 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378305 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378309 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378313 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378317 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378319 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378323 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378326 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378329 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378333 2562 flags.go:64] FLAG: --cgroup-root="" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378336 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378338 2562 flags.go:64] FLAG: --client-ca-file="" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378341 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378344 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378349 2562 flags.go:64] FLAG: --cluster-domain="" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378352 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378354 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 29 15:01:16.383575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378357 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378361 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378364 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378367 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378370 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378373 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378376 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378379 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378382 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378387 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378390 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378394 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378397 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378400 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378402 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378407 2562 flags.go:64] FLAG: --enable-server="true" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378411 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378415 2562 flags.go:64] FLAG: --event-burst="100" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378418 2562 flags.go:64] FLAG: --event-qps="50" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378421 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378423 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378426 2562 flags.go:64] FLAG: --eviction-hard="" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378430 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378432 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378435 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 29 15:01:16.384184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378438 2562 flags.go:64] FLAG: --eviction-soft="" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378441 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378444 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378447 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378450 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378453 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378455 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378458 2562 flags.go:64] FLAG: --feature-gates="" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378462 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378464 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378467 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378470 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378473 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378476 2562 flags.go:64] FLAG: --help="false" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378479 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378482 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378486 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378489 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378492 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378495 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378498 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378501 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378506 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 29 15:01:16.384816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378509 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378512 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378515 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378519 2562 flags.go:64] FLAG: --kube-reserved="" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378522 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378525 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378527 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378530 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378533 2562 flags.go:64] FLAG: --lock-file="" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378536 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378538 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378541 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378546 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378549 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378552 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378554 2562 flags.go:64] FLAG: --logging-format="text" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378557 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378560 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378563 2562 flags.go:64] FLAG: --manifest-url="" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378566 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378570 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378573 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378577 2562 flags.go:64] FLAG: --max-pods="110" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378580 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378583 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 29 15:01:16.385425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378587 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378590 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378607 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378610 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378613 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378620 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378625 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378628 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378631 2562 flags.go:64] FLAG: --pod-cidr="" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378636 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378642 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378645 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378648 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378651 2562 flags.go:64] FLAG: --port="10250" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378654 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378657 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a763fba2b16dc51b" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378660 2562 flags.go:64] FLAG: --qos-reserved="" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378663 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378665 2562 flags.go:64] FLAG: --register-node="true" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378668 2562 flags.go:64] FLAG: --register-with-taints="" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378671 2562 flags.go:64] FLAG: --registry-burst="10" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378674 2562 flags.go:64] FLAG: --registry-qps="5" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378677 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378679 2562 flags.go:64] FLAG: --reserved-memory="" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378683 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 29 15:01:16.386042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378686 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378689 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378691 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378694 2562 flags.go:64] FLAG: --runonce="false" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378697 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378700 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378703 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378706 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378709 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378712 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378715 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378718 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378721 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378724 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378728 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378730 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378733 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378736 2562 flags.go:64] FLAG: --system-cgroups="" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378739 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378745 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378748 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378751 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378755 2562 flags.go:64] FLAG: --tls-min-version="" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378758 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378761 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 29 15:01:16.386670 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378763 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378766 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378769 2562 flags.go:64] FLAG: --v="2" Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378773 2562 flags.go:64] FLAG: --version="false" Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378777 2562 flags.go:64] FLAG: --vmodule="" Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378781 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.378784 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378902 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378906 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378909 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378912 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378915 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378918 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378921 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378924 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378927 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378930 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378933 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378936 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378938 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378941 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 15:01:16.387256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378947 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378950 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378953 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378955 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378959 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378961 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378964 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378967 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378969 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378972 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378974 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378977 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378980 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378982 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378985 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378987 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378990 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378992 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378994 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378997 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 15:01:16.387795 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.378999 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379002 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379004 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379007 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379009 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379013 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379015 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379018 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379020 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379023 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379025 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379028 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379031 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379034 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379037 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379039 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379042 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379045 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379047 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 15:01:16.388276 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379051 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379055 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379058 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379060 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379063 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379065 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379068 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379070 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379072 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379075 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379078 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379080 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379082 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379085 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379087 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379090 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379093 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379095 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379099 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379101 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 15:01:16.388750 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379104 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379106 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379109 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379111 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379113 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379117 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379119 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379122 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379125 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379127 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379130 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379132 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379136 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379139 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379142 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379145 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379148 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379151 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379153 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 15:01:16.389220 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379156 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379159 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379161 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379164 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379166 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379170 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.379173 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.379179 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.386455 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.386470 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.387221 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.387228 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387294 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387299 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 15:01:16.389700 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387302 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387305 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387311 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387315 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387317 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387320 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387323 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387325 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387328 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387331 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387340 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387343 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387345 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387348 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387350 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387353 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387355 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387358 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387361 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387363 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 15:01:16.390051 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387366 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387368 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387371 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387373 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387376 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387380 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387384 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387387 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387391 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387394 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387396 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387400 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387404 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387407 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387410 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387412 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387414 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387417 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387419 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 15:01:16.390587 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387422 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387424 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387427 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387429 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387437 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387440 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387442 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387445 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387447 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387450 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387452 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387455 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387457 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387460 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387462 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387465 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387467 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387470 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387472 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 15:01:16.391200 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387475 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387478 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387481 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387484 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387486 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387488 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387491 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387494 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387496 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387498 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387501 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387503 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387505 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387508 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387510 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387513 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387515 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387518 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387525 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387528 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 15:01:16.391673 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387531 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387533 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387537 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387540 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387543 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387546 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387549 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387551 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387554 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387557 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387559 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387562 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387565 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387568 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387571 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387573 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387576 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387578 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 15:01:16.392175 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387581 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.387586 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387766 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387771 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387774 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387777 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387780 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387783 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387786 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387788 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387791 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387793 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387796 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387805 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387808 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 15:01:16.392618 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387811 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387813 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387816 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387818 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387820 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387823 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387825 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387827 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387830 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387832 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387835 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387837 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387841 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387843 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387846 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387849 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387851 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387853 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387856 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387858 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 15:01:16.393052 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387862 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387865 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387868 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387871 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387874 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387876 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387879 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387882 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387884 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387886 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387889 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387897 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387900 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387902 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387905 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387907 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387910 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387912 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387914 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387917 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 15:01:16.393527 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387920 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387922 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387924 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387927 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387930 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387933 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387935 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387938 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387941 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387943 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387945 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387948 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387951 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387953 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387955 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387958 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387960 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387963 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387965 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387968 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 15:01:16.394028 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387970 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387972 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387975 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387977 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387985 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387988 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387990 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387993 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387995 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.387997 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388000 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388002 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388005 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388008 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388011 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388014 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388017 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388019 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388022 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 15:01:16.394509 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388024 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388028 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388031 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388034 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388036 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388039 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:16.388041 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.388046 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.388168 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.390255 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.391720 2562 server.go:1019] "Starting client certificate rotation" Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.391841 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 15:01:16.395050 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.391884 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 15:01:16.395368 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.395284 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 15:01:16.400181 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.400155 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 15:01:16.409859 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.409840 2562 log.go:25] "Validated CRI v1 runtime API" Jul 29 15:01:16.411858 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.411843 2562 log.go:25] "Validated CRI v1 image API" Jul 29 15:01:16.413420 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.413403 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 29 15:01:16.416656 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.416637 2562 fs.go:135] Filesystem UUIDs: map[23933927-6a32-48c1-8cbf-c257f8f91bcd:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e1aa78d6-f08b-43e7-a6f2-4a9ce9d0e5e0:/dev/nvme0n1p3] Jul 29 15:01:16.416730 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.416656 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 29 15:01:16.422085 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.421971 2562 manager.go:216] Machine: {Timestamp:2026-07-29 15:01:16.420812252 +0000 UTC m=+0.329660624 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3135217 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2358abb13737e646f71f1caddacafa SystemUUID:ec2358ab-b137-37e6-46f7-1f1caddacafa BootID:8a2fdb54-80c8-4122-8a6d-13fa87b85ee8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d2:a8:89:22:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d2:a8:89:22:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:69:9c:a0:a6:a9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 29 15:01:16.422085 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.422078 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 29 15:01:16.422196 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.422159 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 29 15:01:16.423477 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.423460 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 15:01:16.425674 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.425643 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 29 15:01:16.425821 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.425678 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-104.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 29 15:01:16.426510 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.426500 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 29 15:01:16.426539 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.426513 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 29 15:01:16.426539 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.426520 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 15:01:16.427006 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.426996 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 15:01:16.427763 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.427754 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 29 15:01:16.429531 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.429520 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 29 15:01:16.429706 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.429696 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 29 15:01:16.429740 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.429733 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 29 15:01:16.429774 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.429750 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 29 15:01:16.429805 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.429775 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 29 15:01:16.429805 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.429784 2562 kubelet.go:398] "Adding apiserver pod source" Jul 29 15:01:16.429805 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.429793 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 29 15:01:16.430232 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.430220 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 15:01:16.430273 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.430239 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 15:01:16.432676 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.432659 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 29 15:01:16.434684 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.434667 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 29 15:01:16.434773 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.434706 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 29 15:01:16.436564 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436550 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 29 15:01:16.436643 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436570 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 29 15:01:16.436643 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436579 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 29 15:01:16.436643 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436588 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 29 15:01:16.436643 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436617 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 29 15:01:16.436643 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436626 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 29 15:01:16.436643 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436638 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 29 15:01:16.436862 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436647 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 29 15:01:16.436862 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436656 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 29 15:01:16.436862 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436665 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 29 15:01:16.436862 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436676 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 29 15:01:16.436862 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.436685 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 29 15:01:16.437358 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.437346 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 29 15:01:16.437403 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.437361 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 29 15:01:16.440539 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.440524 2562 server.go:1268] "Started kubelet" Jul 29 15:01:16.440697 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.440623 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 29 15:01:16.440792 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.440712 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 29 15:01:16.440792 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.440716 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 29 15:01:16.441347 ip-10-0-142-104 systemd[1]: Started Kubernetes Kubelet. Jul 29 15:01:16.444493 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.443809 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 29 15:01:16.446503 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.446068 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 29 15:01:16.446503 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.446143 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-104.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 29 15:01:16.446503 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.446307 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 29 15:01:16.446929 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.446899 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 29 15:01:16.448882 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.448860 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 29 15:01:16.450898 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.449913 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-104.ec2.internal.18c6cacb367fc7ae default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-104.ec2.internal,UID:ip-10-0-142-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-104.ec2.internal,},FirstTimestamp:2026-07-29 15:01:16.440496046 +0000 UTC m=+0.349344417,LastTimestamp:2026-07-29 15:01:16.440496046 +0000 UTC m=+0.349344417,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-104.ec2.internal,}" Jul 29 15:01:16.454535 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.454516 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 29 15:01:16.454649 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.454531 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 29 15:01:16.454713 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.454676 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-104.ec2.internal\" not found" Jul 29 15:01:16.455086 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.455065 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 29 15:01:16.455086 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.455078 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 29 15:01:16.455223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.455084 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 29 15:01:16.455223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.455196 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 29 15:01:16.455309 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.455225 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 29 15:01:16.455537 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.455521 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 29 15:01:16.455537 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.455537 2562 factory.go:55] Registering systemd factory Jul 29 15:01:16.455748 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.455546 2562 factory.go:223] Registration of the systemd container factory successfully Jul 29 15:01:16.456097 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.456082 2562 factory.go:153] Registering CRI-O factory Jul 29 15:01:16.456097 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.456099 2562 factory.go:223] Registration of the crio container factory successfully Jul 29 15:01:16.456216 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.456122 2562 factory.go:103] Registering Raw factory Jul 29 15:01:16.456216 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.456134 2562 manager.go:1196] Started watching for new ooms in manager Jul 29 15:01:16.459372 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.459354 2562 manager.go:349] Starting recovery of all containers Jul 29 15:01:16.459548 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.459524 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 29 15:01:16.459638 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.459589 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 29 15:01:16.469105 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.469086 2562 manager.go:354] Recovery completed Jul 29 15:01:16.473314 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.473301 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 15:01:16.473742 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.473726 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gzhkf" Jul 29 15:01:16.475869 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.475848 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientMemory" Jul 29 15:01:16.475934 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.475874 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 15:01:16.475934 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.475885 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientPID" Jul 29 15:01:16.476450 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.476431 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 29 15:01:16.476450 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.476439 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 15:01:16.476450 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.476446 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 29 15:01:16.476647 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.476461 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 29 15:01:16.477485 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.477469 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientMemory" Jul 29 15:01:16.477538 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.477497 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 15:01:16.477538 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.477510 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientPID" Jul 29 15:01:16.477641 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.477439 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-104.ec2.internal.18c6cacb389b6844 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-104.ec2.internal,UID:ip-10-0-142-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-104.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-104.ec2.internal,},FirstTimestamp:2026-07-29 15:01:16.47586106 +0000 UTC m=+0.384709431,LastTimestamp:2026-07-29 15:01:16.47586106 +0000 UTC m=+0.384709431,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-104.ec2.internal,}" Jul 29 15:01:16.478699 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.478684 2562 policy_none.go:49] "None policy: Start" Jul 29 15:01:16.478753 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.478705 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 29 15:01:16.478753 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.478718 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 29 15:01:16.479763 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.479753 2562 policy_none.go:47] "Start" Jul 29 15:01:16.481353 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.481218 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gzhkf" Jul 29 15:01:16.486166 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.486093 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-104.ec2.internal.18c6cacb389baf87 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-104.ec2.internal,UID:ip-10-0-142-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-104.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-104.ec2.internal,},FirstTimestamp:2026-07-29 15:01:16.475879303 +0000 UTC m=+0.384727674,LastTimestamp:2026-07-29 15:01:16.475879303 +0000 UTC m=+0.384727674,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-104.ec2.internal,}" Jul 29 15:01:16.515334 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.515308 2562 manager.go:337] "Starting Device Plugin manager" Jul 29 15:01:16.533555 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.515373 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 29 15:01:16.533555 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.515390 2562 server.go:87] "Starting device plugin registration server" Jul 29 15:01:16.533555 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.515732 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 29 15:01:16.533555 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.515747 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 29 15:01:16.533555 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.515842 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 29 15:01:16.533555 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.515917 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 29 15:01:16.533555 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.515926 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 29 15:01:16.533555 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.516498 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 29 15:01:16.533555 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.516553 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-104.ec2.internal\" not found" Jul 29 15:01:16.561611 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.561553 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 29 15:01:16.562891 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.562868 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 29 15:01:16.562964 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.562920 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 29 15:01:16.562964 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.562947 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 29 15:01:16.563050 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.562989 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 29 15:01:16.565506 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.565484 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 15:01:16.616909 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.616876 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 15:01:16.618625 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.618586 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientMemory" Jul 29 15:01:16.618625 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.618631 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 15:01:16.618776 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.618642 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientPID" Jul 29 15:01:16.618776 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.618666 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.631549 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.631501 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.631549 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.631526 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-104.ec2.internal\": node \"ip-10-0-142-104.ec2.internal\" not found" Jul 29 15:01:16.663353 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.663335 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal"] Jul 29 15:01:16.663448 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.663395 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 15:01:16.664176 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.664160 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientMemory" Jul 29 15:01:16.664233 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.664195 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 15:01:16.664233 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.664205 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientPID" Jul 29 15:01:16.665470 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.665459 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 15:01:16.665634 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.665617 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.665674 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.665652 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 15:01:16.666174 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.666150 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientMemory" Jul 29 15:01:16.666174 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.666161 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientMemory" Jul 29 15:01:16.666303 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.666179 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 15:01:16.666303 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.666192 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientPID" Jul 29 15:01:16.666303 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.666182 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 15:01:16.666303 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.666269 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientPID" Jul 29 15:01:16.667763 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.667748 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.667811 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.667783 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 15:01:16.668467 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.668454 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientMemory" Jul 29 15:01:16.668541 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.668479 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 15:01:16.668541 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.668489 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeHasSufficientPID" Jul 29 15:01:16.684185 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.684163 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-104.ec2.internal\" not found" Jul 29 15:01:16.692429 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.692406 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-104.ec2.internal\" not found" node="ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.696945 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.696929 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-104.ec2.internal\" not found" node="ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.756513 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.756485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b336e36e39f016f7f0e11ef816392a81-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal\" (UID: \"b336e36e39f016f7f0e11ef816392a81\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.756513 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.756515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c19a4e2c03ee49097a3c2455ba7ff61f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-104.ec2.internal\" (UID: \"c19a4e2c03ee49097a3c2455ba7ff61f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.756687 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.756531 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b336e36e39f016f7f0e11ef816392a81-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal\" (UID: \"b336e36e39f016f7f0e11ef816392a81\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.784372 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.784346 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-104.ec2.internal\" not found" Jul 29 15:01:16.857253 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.857222 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c19a4e2c03ee49097a3c2455ba7ff61f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-104.ec2.internal\" (UID: \"c19a4e2c03ee49097a3c2455ba7ff61f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.857331 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.857259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b336e36e39f016f7f0e11ef816392a81-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal\" (UID: \"b336e36e39f016f7f0e11ef816392a81\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.857331 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.857278 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b336e36e39f016f7f0e11ef816392a81-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal\" (UID: \"b336e36e39f016f7f0e11ef816392a81\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.857392 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.857324 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b336e36e39f016f7f0e11ef816392a81-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal\" (UID: \"b336e36e39f016f7f0e11ef816392a81\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.857392 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.857330 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c19a4e2c03ee49097a3c2455ba7ff61f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-104.ec2.internal\" (UID: \"c19a4e2c03ee49097a3c2455ba7ff61f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.857392 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.857387 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b336e36e39f016f7f0e11ef816392a81-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal\" (UID: \"b336e36e39f016f7f0e11ef816392a81\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" Jul 29 15:01:16.885406 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.885342 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-104.ec2.internal\" not found" Jul 29 15:01:16.986247 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:16.986218 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-104.ec2.internal\" not found" Jul 29 15:01:16.995422 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:16.995386 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" Jul 29 15:01:17.000486 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.000466 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal" Jul 29 15:01:17.086940 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.086898 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-104.ec2.internal\" not found" Jul 29 15:01:17.187537 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.187477 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-104.ec2.internal\" not found" Jul 29 15:01:17.287957 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.287932 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-104.ec2.internal\" not found" Jul 29 15:01:17.362033 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.362012 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 15:01:17.391238 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.391216 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 29 15:01:17.391733 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.391315 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 15:01:17.391733 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.391354 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 15:01:17.430557 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.430533 2562 apiserver.go:52] "Watching apiserver" Jul 29 15:01:17.448262 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.448241 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 29 15:01:17.449194 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.449176 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-2wrlr","openshift-network-operator/iptables-alerter-87r9f","openshift-ovn-kubernetes/ovnkube-node-cmhbw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp","openshift-dns/node-resolver-pp8jk","openshift-image-registry/node-ca-rthsh","openshift-multus/multus-npfvr","kube-system/global-pull-secret-syncer-qc8sd","kube-system/konnectivity-agent-2brl7","openshift-cluster-node-tuning-operator/tuned-lzwd6","openshift-multus/multus-additional-cni-plugins-ftrpv","openshift-multus/network-metrics-daemon-v4svt"] Jul 29 15:01:17.450510 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.450486 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:17.450624 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.450570 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:17.451637 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.451612 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.456833 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.453247 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.456833 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.455527 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" Jul 29 15:01:17.456833 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.455652 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 29 15:01:17.456833 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.456076 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.457692 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.457663 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 29 15:01:17.457808 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.457710 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 29 15:01:17.457808 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.457764 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 29 15:01:17.457912 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.457845 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 29 15:01:17.457912 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.457867 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 29 15:01:17.457912 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.457888 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 29 15:01:17.458053 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.457963 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 29 15:01:17.458053 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.458000 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.458147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.458052 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tdxr4\"" Jul 29 15:01:17.458220 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.458181 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j5tlb\"" Jul 29 15:01:17.458281 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.458249 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 29 15:01:17.458422 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.458401 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 29 15:01:17.458520 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.458485 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 29 15:01:17.458589 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.458524 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z7qxr\"" Jul 29 15:01:17.458589 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.458578 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 29 15:01:17.458719 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.458620 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 29 15:01:17.458917 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.458902 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.460247 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460228 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.460662 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-cni-netd\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.460776 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460673 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-ovnkube-config\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.460776 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460698 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-device-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.460776 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-ovn-node-metrics-cert\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.460962 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460780 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 29 15:01:17.460962 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460789 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfvwr\" (UniqueName: \"kubernetes.io/projected/fd305d57-301d-47e9-87cb-f664d2a9c6f1-kube-api-access-zfvwr\") pod \"iptables-alerter-87r9f\" (UID: \"fd305d57-301d-47e9-87cb-f664d2a9c6f1\") " pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.460962 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460800 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-79fks\"" Jul 29 15:01:17.460962 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460827 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-cni-bin\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.460962 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460857 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 29 15:01:17.460962 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-env-overrides\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.460962 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-socket-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.460962 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-slash\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.460962 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-run-ovn-kubernetes\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.460986 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-run-netns\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461001 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-run-openvswitch\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461035 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-node-log\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fpqz\" (UniqueName: \"kubernetes.io/projected/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-kube-api-access-7fpqz\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-run-systemd\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461119 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-var-lib-openvswitch\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461143 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6rc6\" (UniqueName: \"kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6\") pod \"network-check-target-2wrlr\" (UID: \"eeb6b594-9641-4f08-875d-6c6d1eb68cb1\") " pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461165 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-run-ovn\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461186 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461209 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-registration-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461234 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-etc-selinux\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461258 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kh27\" (UniqueName: \"kubernetes.io/projected/e3302ec9-0215-4448-b3c0-9b93ba589d31-kube-api-access-4kh27\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.461290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461276 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-etc-openvswitch\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461296 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-log-socket\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461322 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-ovnkube-script-lib\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461370 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd305d57-301d-47e9-87cb-f664d2a9c6f1-iptables-alerter-script\") pod \"iptables-alerter-87r9f\" (UID: \"fd305d57-301d-47e9-87cb-f664d2a9c6f1\") " pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461393 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd305d57-301d-47e9-87cb-f664d2a9c6f1-host-slash\") pod \"iptables-alerter-87r9f\" (UID: \"fd305d57-301d-47e9-87cb-f664d2a9c6f1\") " pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461437 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461435 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-kubelet\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461541 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-sys-fs\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461572 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-systemd-units\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.461723 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:17.461870 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.461779 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:17.462877 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.462860 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 29 15:01:17.463020 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.462875 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 29 15:01:17.463020 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.462915 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 29 15:01:17.463020 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.462969 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:17.463020 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.462863 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 29 15:01:17.463260 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.463052 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lmj7v\"" Jul 29 15:01:17.463260 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.463116 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 29 15:01:17.463260 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.463134 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 29 15:01:17.463386 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.463293 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n9crl\"" Jul 29 15:01:17.464298 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.464282 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.464739 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.464723 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 29 15:01:17.465078 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.465063 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 29 15:01:17.465157 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.465110 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p8xg7\"" Jul 29 15:01:17.465379 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.465364 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 15:01:17.465842 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.465824 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.466110 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.466094 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 29 15:01:17.466200 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.466180 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6slwk\"" Jul 29 15:01:17.466359 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.466345 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 29 15:01:17.467118 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.467097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:17.467197 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.467168 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:17.467844 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.467827 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n7sxz\"" Jul 29 15:01:17.467916 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.467863 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 29 15:01:17.467916 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.467834 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 29 15:01:17.471782 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.471765 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 15:01:17.471870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.471833 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal" Jul 29 15:01:17.471947 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.471932 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal"] Jul 29 15:01:17.479779 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.479761 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal"] Jul 29 15:01:17.479942 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.479927 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 15:01:17.485349 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.485283 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 14:56:16 +0000 UTC" deadline="2028-04-07 08:40:55.382547 +0000 UTC" Jul 29 15:01:17.485439 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.485350 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14825h39m37.89720429s" Jul 29 15:01:17.503190 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.503171 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-446bp" Jul 29 15:01:17.512630 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.512612 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-446bp" Jul 29 15:01:17.538753 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:17.538730 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb336e36e39f016f7f0e11ef816392a81.slice/crio-e45f82586d7cfc50293da6c5a52c0bf8846dd1772432bd41f7c21093456019d5 WatchSource:0}: Error finding container e45f82586d7cfc50293da6c5a52c0bf8846dd1772432bd41f7c21093456019d5: Status 404 returned error can't find the container with id e45f82586d7cfc50293da6c5a52c0bf8846dd1772432bd41f7c21093456019d5 Jul 29 15:01:17.543475 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.543453 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 15:01:17.556099 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.556081 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 29 15:01:17.561726 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-cni-bin\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.561809 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-system-cni-dir\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.561809 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561765 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shpnm\" (UniqueName: \"kubernetes.io/projected/c9a1ce69-6788-4461-a3f8-e36f039aa692-kube-api-access-shpnm\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.561809 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561782 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-cni-bin\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.561934 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561804 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30d268aa-dda6-492b-acd6-b14a838c5948-serviceca\") pod \"node-ca-rthsh\" (UID: \"30d268aa-dda6-492b-acd6-b14a838c5948\") " pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.561934 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e7b52735-d789-4074-9c97-2e5bdb782caa-kubelet-config\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:17.561934 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561855 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-slash\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.561934 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-os-release\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.561934 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561885 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-sysctl-conf\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.561934 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561907 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9a1ce69-6788-4461-a3f8-e36f039aa692-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.562142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561929 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-slash\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561942 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-run-netns\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561971 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fpqz\" (UniqueName: \"kubernetes.io/projected/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-kube-api-access-7fpqz\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.561996 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-cni-binary-copy\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.562142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562018 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-run-netns\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-var-lib-kubelet\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.562142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562059 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c20ebebb-b1b8-4d52-b617-b6faa3d4bf13-hosts-file\") pod \"node-resolver-pp8jk\" (UID: \"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13\") " pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.562142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562085 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:17.562142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562124 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-var-lib-cni-bin\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562155 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-daemon-config\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562182 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-run-systemd\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kh27\" (UniqueName: \"kubernetes.io/projected/e3302ec9-0215-4448-b3c0-9b93ba589d31-kube-api-access-4kh27\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562261 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-run-systemd\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562255 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-cnibin\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-os-release\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562287 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562327 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6zh5\" (UniqueName: \"kubernetes.io/projected/30d268aa-dda6-492b-acd6-b14a838c5948-kube-api-access-k6zh5\") pod \"node-ca-rthsh\" (UID: \"30d268aa-dda6-492b-acd6-b14a838c5948\") " pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562358 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-log-socket\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd305d57-301d-47e9-87cb-f664d2a9c6f1-host-slash\") pod \"iptables-alerter-87r9f\" (UID: \"fd305d57-301d-47e9-87cb-f664d2a9c6f1\") " pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-kubelet\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562430 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562430 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-log-socket\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562438 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-kubelet\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562448 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd305d57-301d-47e9-87cb-f664d2a9c6f1-host-slash\") pod \"iptables-alerter-87r9f\" (UID: \"fd305d57-301d-47e9-87cb-f664d2a9c6f1\") " pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.562517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562466 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-sys-fs\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562484 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562491 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-lib-modules\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562517 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-system-cni-dir\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562531 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-sys-fs\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-systemd-units\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562580 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-cni-netd\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562614 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-ovnkube-config\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-cni-dir\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562650 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-cni-netd\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562676 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/22bad190-4a5f-47f7-a261-25ec14e59621-agent-certs\") pod \"konnectivity-agent-2brl7\" (UID: \"22bad190-4a5f-47f7-a261-25ec14e59621\") " pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562696 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-systemd-units\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w8nw\" (UniqueName: \"kubernetes.io/projected/6c883406-1472-4688-819f-462bbb78df59-kube-api-access-6w8nw\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562735 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-run\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e7b52735-d789-4074-9c97-2e5bdb782caa-dbus\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-env-overrides\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.563275 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562829 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-socket-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562888 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-run-k8s-cni-cncf-io\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/22bad190-4a5f-47f7-a261-25ec14e59621-konnectivity-ca\") pod \"konnectivity-agent-2brl7\" (UID: \"22bad190-4a5f-47f7-a261-25ec14e59621\") " pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562950 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c52f73-7c5d-4452-96bc-4217e5341a27-tmp\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.562983 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9a1ce69-6788-4461-a3f8-e36f039aa692-cni-binary-copy\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-run-ovn-kubernetes\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563024 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-socket-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-hostroot\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563070 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-host-run-ovn-kubernetes\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563077 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-sysctl-d\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563140 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-tuned\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563160 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-ovnkube-config\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563180 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-run-openvswitch\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563180 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-env-overrides\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563267 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-node-log\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563303 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-socket-dir-parent\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563322 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-run-netns\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.564028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-modprobe-d\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-kubernetes\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563370 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-run-openvswitch\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-var-lib-openvswitch\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563402 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rc6\" (UniqueName: \"kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6\") pod \"network-check-target-2wrlr\" (UID: \"eeb6b594-9641-4f08-875d-6c6d1eb68cb1\") " pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-run-ovn\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563424 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-node-log\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563439 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-registration-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-etc-selinux\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563494 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-etc-openvswitch\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563507 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-var-lib-openvswitch\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563535 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-run-ovn\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563549 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-etc-selinux\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563572 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-registration-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-ovnkube-script-lib\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-var-lib-cni-multus\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563663 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-etc-openvswitch\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.564735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563701 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrgbz\" (UniqueName: \"kubernetes.io/projected/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-kube-api-access-qrgbz\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563733 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-sysconfig\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563841 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9a1ce69-6788-4461-a3f8-e36f039aa692-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd305d57-301d-47e9-87cb-f664d2a9c6f1-iptables-alerter-script\") pod \"iptables-alerter-87r9f\" (UID: \"fd305d57-301d-47e9-87cb-f664d2a9c6f1\") " pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-host\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7dsk\" (UniqueName: \"kubernetes.io/projected/c20ebebb-b1b8-4d52-b617-b6faa3d4bf13-kube-api-access-w7dsk\") pod \"node-resolver-pp8jk\" (UID: \"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13\") " pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30d268aa-dda6-492b-acd6-b14a838c5948-host\") pod \"node-ca-rthsh\" (UID: \"30d268aa-dda6-492b-acd6-b14a838c5948\") " pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563971 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-device-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.563992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-sys\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564015 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c20ebebb-b1b8-4d52-b617-b6faa3d4bf13-tmp-dir\") pod \"node-resolver-pp8jk\" (UID: \"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13\") " pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564038 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-ovn-node-metrics-cert\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564058 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-cnibin\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564082 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-conf-dir\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-run-multus-certs\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564128 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-etc-kubernetes\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564129 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e3302ec9-0215-4448-b3c0-9b93ba589d31-device-dir\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.565519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564152 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-ovnkube-script-lib\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.566165 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564177 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-systemd\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.566165 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564219 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-var-lib-kubelet\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.566165 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564246 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5h5d\" (UniqueName: \"kubernetes.io/projected/24c52f73-7c5d-4452-96bc-4217e5341a27-kube-api-access-w5h5d\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.566165 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564277 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfvwr\" (UniqueName: \"kubernetes.io/projected/fd305d57-301d-47e9-87cb-f664d2a9c6f1-kube-api-access-zfvwr\") pod \"iptables-alerter-87r9f\" (UID: \"fd305d57-301d-47e9-87cb-f664d2a9c6f1\") " pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.566165 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564407 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd305d57-301d-47e9-87cb-f664d2a9c6f1-iptables-alerter-script\") pod \"iptables-alerter-87r9f\" (UID: \"fd305d57-301d-47e9-87cb-f664d2a9c6f1\") " pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.566165 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.564513 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 29 15:01:17.566633 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.566560 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" event={"ID":"b336e36e39f016f7f0e11ef816392a81","Type":"ContainerStarted","Data":"e45f82586d7cfc50293da6c5a52c0bf8846dd1772432bd41f7c21093456019d5"} Jul 29 15:01:17.568154 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.568137 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-ovn-node-metrics-cert\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.570713 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.570696 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 15:01:17.570713 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.570714 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 15:01:17.570844 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.570729 2562 projected.go:196] Error preparing data for projected volume kube-api-access-n6rc6 for pod openshift-network-diagnostics/network-check-target-2wrlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:17.570844 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.570790 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6 podName:eeb6b594-9641-4f08-875d-6c6d1eb68cb1 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:18.070774085 +0000 UTC m=+1.979622470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n6rc6" (UniqueName: "kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6") pod "network-check-target-2wrlr" (UID: "eeb6b594-9641-4f08-875d-6c6d1eb68cb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:17.572358 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.572337 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kh27\" (UniqueName: \"kubernetes.io/projected/e3302ec9-0215-4448-b3c0-9b93ba589d31-kube-api-access-4kh27\") pod \"aws-ebs-csi-driver-node-vcwvp\" (UID: \"e3302ec9-0215-4448-b3c0-9b93ba589d31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.572845 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.572830 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfvwr\" (UniqueName: \"kubernetes.io/projected/fd305d57-301d-47e9-87cb-f664d2a9c6f1-kube-api-access-zfvwr\") pod \"iptables-alerter-87r9f\" (UID: \"fd305d57-301d-47e9-87cb-f664d2a9c6f1\") " pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.573837 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.573733 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fpqz\" (UniqueName: \"kubernetes.io/projected/4e087d29-d6fa-400e-a408-7d0a2cf8b6f3-kube-api-access-7fpqz\") pod \"ovnkube-node-cmhbw\" (UID: \"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.595958 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.595936 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 15:01:17.616358 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.616333 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 15:01:17.664950 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.664922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-lib-modules\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.664950 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.664954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-system-cni-dir\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.664972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.664987 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-cni-dir\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/22bad190-4a5f-47f7-a261-25ec14e59621-agent-certs\") pod \"konnectivity-agent-2brl7\" (UID: \"22bad190-4a5f-47f7-a261-25ec14e59621\") " pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665019 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w8nw\" (UniqueName: \"kubernetes.io/projected/6c883406-1472-4688-819f-462bbb78df59-kube-api-access-6w8nw\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665042 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-run\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665042 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-system-cni-dir\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e7b52735-d789-4074-9c97-2e5bdb782caa-dbus\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665083 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-run-k8s-cni-cncf-io\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665102 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/22bad190-4a5f-47f7-a261-25ec14e59621-konnectivity-ca\") pod \"konnectivity-agent-2brl7\" (UID: \"22bad190-4a5f-47f7-a261-25ec14e59621\") " pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.665114 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c52f73-7c5d-4452-96bc-4217e5341a27-tmp\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.665147 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665151 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9a1ce69-6788-4461-a3f8-e36f039aa692-cni-binary-copy\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665168 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e7b52735-d789-4074-9c97-2e5bdb782caa-dbus\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665172 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-cni-dir\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.665187 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs podName:6c883406-1472-4688-819f-462bbb78df59 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:18.165165917 +0000 UTC m=+2.074014279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs") pod "network-metrics-daemon-v4svt" (UID: "6c883406-1472-4688-819f-462bbb78df59") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665114 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-run\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665116 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-lib-modules\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-hostroot\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-sysctl-d\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-tuned\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665291 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-hostroot\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665295 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-socket-dir-parent\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665327 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-run-netns\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665344 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-socket-dir-parent\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665352 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-modprobe-d\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-kubernetes\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665424 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-var-lib-cni-multus\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665432 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-modprobe-d\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.665781 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665426 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-sysctl-d\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrgbz\" (UniqueName: \"kubernetes.io/projected/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-kube-api-access-qrgbz\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665468 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-run-netns\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665481 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-sysconfig\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665477 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-var-lib-cni-multus\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665516 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665523 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-sysconfig\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665544 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-kubernetes\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665548 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9a1ce69-6788-4461-a3f8-e36f039aa692-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665712 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-host\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7dsk\" (UniqueName: \"kubernetes.io/projected/c20ebebb-b1b8-4d52-b617-b6faa3d4bf13-kube-api-access-w7dsk\") pod \"node-resolver-pp8jk\" (UID: \"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13\") " pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30d268aa-dda6-492b-acd6-b14a838c5948-host\") pod \"node-ca-rthsh\" (UID: \"30d268aa-dda6-492b-acd6-b14a838c5948\") " pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665801 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9a1ce69-6788-4461-a3f8-e36f039aa692-cni-binary-copy\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-sys\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c20ebebb-b1b8-4d52-b617-b6faa3d4bf13-tmp-dir\") pod \"node-resolver-pp8jk\" (UID: \"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13\") " pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665895 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-cnibin\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-conf-dir\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.666592 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.665941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-run-multus-certs\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666027 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-sys\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666068 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30d268aa-dda6-492b-acd6-b14a838c5948-host\") pod \"node-ca-rthsh\" (UID: \"30d268aa-dda6-492b-acd6-b14a838c5948\") " pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666089 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9a1ce69-6788-4461-a3f8-e36f039aa692-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666116 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-host\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666169 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-cnibin\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-etc-kubernetes\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666215 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-systemd\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-conf-dir\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666244 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-var-lib-kubelet\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5h5d\" (UniqueName: \"kubernetes.io/projected/24c52f73-7c5d-4452-96bc-4217e5341a27-kube-api-access-w5h5d\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666290 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-etc-kubernetes\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666300 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-system-cni-dir\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shpnm\" (UniqueName: \"kubernetes.io/projected/c9a1ce69-6788-4461-a3f8-e36f039aa692-kube-api-access-shpnm\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666335 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-run-multus-certs\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30d268aa-dda6-492b-acd6-b14a838c5948-serviceca\") pod \"node-ca-rthsh\" (UID: \"30d268aa-dda6-492b-acd6-b14a838c5948\") " pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e7b52735-d789-4074-9c97-2e5bdb782caa-kubelet-config\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-os-release\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.667579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666432 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c20ebebb-b1b8-4d52-b617-b6faa3d4bf13-tmp-dir\") pod \"node-resolver-pp8jk\" (UID: \"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13\") " pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666435 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-sysctl-conf\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9a1ce69-6788-4461-a3f8-e36f039aa692-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-cni-binary-copy\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666548 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-var-lib-kubelet\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666558 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-sysctl-conf\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666573 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c20ebebb-b1b8-4d52-b617-b6faa3d4bf13-hosts-file\") pod \"node-resolver-pp8jk\" (UID: \"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13\") " pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666615 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-var-lib-kubelet\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666619 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666635 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e7b52735-d789-4074-9c97-2e5bdb782caa-kubelet-config\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-var-lib-cni-bin\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666686 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-daemon-config\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666696 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-os-release\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666464 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-systemd\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666716 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-cnibin\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-os-release\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6zh5\" (UniqueName: \"kubernetes.io/projected/30d268aa-dda6-492b-acd6-b14a838c5948-kube-api-access-k6zh5\") pod \"node-ca-rthsh\" (UID: \"30d268aa-dda6-492b-acd6-b14a838c5948\") " pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.668235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666920 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30d268aa-dda6-492b-acd6-b14a838c5948-serviceca\") pod \"node-ca-rthsh\" (UID: \"30d268aa-dda6-492b-acd6-b14a838c5948\") " pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666975 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-run-k8s-cni-cncf-io\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.666988 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-system-cni-dir\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667043 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-var-lib-cni-bin\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667095 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-cnibin\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667151 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9a1ce69-6788-4461-a3f8-e36f039aa692-os-release\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667153 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9a1ce69-6788-4461-a3f8-e36f039aa692-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.666688 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667205 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-host-var-lib-kubelet\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667221 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c20ebebb-b1b8-4d52-b617-b6faa3d4bf13-hosts-file\") pod \"node-resolver-pp8jk\" (UID: \"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13\") " pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:17.667232 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret podName:e7b52735-d789-4074-9c97-2e5bdb782caa nodeName:}" failed. No retries permitted until 2026-07-29 15:01:18.167215532 +0000 UTC m=+2.076063891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret") pod "global-pull-secret-syncer-qc8sd" (UID: "e7b52735-d789-4074-9c97-2e5bdb782caa") : object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667696 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/22bad190-4a5f-47f7-a261-25ec14e59621-konnectivity-ca\") pod \"konnectivity-agent-2brl7\" (UID: \"22bad190-4a5f-47f7-a261-25ec14e59621\") " pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667824 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-cni-binary-copy\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667835 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-multus-daemon-config\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667871 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c52f73-7c5d-4452-96bc-4217e5341a27-tmp\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.667890 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24c52f73-7c5d-4452-96bc-4217e5341a27-etc-tuned\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.668716 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.668021 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/22bad190-4a5f-47f7-a261-25ec14e59621-agent-certs\") pod \"konnectivity-agent-2brl7\" (UID: \"22bad190-4a5f-47f7-a261-25ec14e59621\") " pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:17.676664 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.676646 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w8nw\" (UniqueName: \"kubernetes.io/projected/6c883406-1472-4688-819f-462bbb78df59-kube-api-access-6w8nw\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:17.678714 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.678691 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrgbz\" (UniqueName: \"kubernetes.io/projected/6800b94b-4d3d-4b5a-ab8b-68083bbd2520-kube-api-access-qrgbz\") pod \"multus-npfvr\" (UID: \"6800b94b-4d3d-4b5a-ab8b-68083bbd2520\") " pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.678938 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.678922 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7dsk\" (UniqueName: \"kubernetes.io/projected/c20ebebb-b1b8-4d52-b617-b6faa3d4bf13-kube-api-access-w7dsk\") pod \"node-resolver-pp8jk\" (UID: \"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13\") " pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.679003 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.678951 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shpnm\" (UniqueName: \"kubernetes.io/projected/c9a1ce69-6788-4461-a3f8-e36f039aa692-kube-api-access-shpnm\") pod \"multus-additional-cni-plugins-ftrpv\" (UID: \"c9a1ce69-6788-4461-a3f8-e36f039aa692\") " pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.679064 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.679049 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6zh5\" (UniqueName: \"kubernetes.io/projected/30d268aa-dda6-492b-acd6-b14a838c5948-kube-api-access-k6zh5\") pod \"node-ca-rthsh\" (UID: \"30d268aa-dda6-492b-acd6-b14a838c5948\") " pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.679317 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.679301 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5h5d\" (UniqueName: \"kubernetes.io/projected/24c52f73-7c5d-4452-96bc-4217e5341a27-kube-api-access-w5h5d\") pod \"tuned-lzwd6\" (UID: \"24c52f73-7c5d-4452-96bc-4217e5341a27\") " pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.783742 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.783665 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-87r9f" Jul 29 15:01:17.792878 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:17.792856 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd305d57_301d_47e9_87cb_f664d2a9c6f1.slice/crio-8c540c14e6dd399c87097185c3850770c9bd5c9bc0d72d0e1e54d7b3a802b0a6 WatchSource:0}: Error finding container 8c540c14e6dd399c87097185c3850770c9bd5c9bc0d72d0e1e54d7b3a802b0a6: Status 404 returned error can't find the container with id 8c540c14e6dd399c87097185c3850770c9bd5c9bc0d72d0e1e54d7b3a802b0a6 Jul 29 15:01:17.796131 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.796096 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:17.802676 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.802653 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" Jul 29 15:01:17.803256 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:17.803239 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e087d29_d6fa_400e_a408_7d0a2cf8b6f3.slice/crio-44d0e1b78aeffd0e5ae6f74dadfbf44fb804a78d30799b48fd3df574dd112586 WatchSource:0}: Error finding container 44d0e1b78aeffd0e5ae6f74dadfbf44fb804a78d30799b48fd3df574dd112586: Status 404 returned error can't find the container with id 44d0e1b78aeffd0e5ae6f74dadfbf44fb804a78d30799b48fd3df574dd112586 Jul 29 15:01:17.810831 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:17.810810 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3302ec9_0215_4448_b3c0_9b93ba589d31.slice/crio-bd5e2a2d26f38d278938d61149339fa7ab8914a6222b94d40fd7872a8e2cc842 WatchSource:0}: Error finding container bd5e2a2d26f38d278938d61149339fa7ab8914a6222b94d40fd7872a8e2cc842: Status 404 returned error can't find the container with id bd5e2a2d26f38d278938d61149339fa7ab8914a6222b94d40fd7872a8e2cc842 Jul 29 15:01:17.845555 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.845518 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pp8jk" Jul 29 15:01:17.851460 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.851443 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rthsh" Jul 29 15:01:17.852835 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:17.852811 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc20ebebb_b1b8_4d52_b617_b6faa3d4bf13.slice/crio-af170d023a0bf46d4197afb3bf1fd0d18415225ff0c267cb6aab4ec371584a2c WatchSource:0}: Error finding container af170d023a0bf46d4197afb3bf1fd0d18415225ff0c267cb6aab4ec371584a2c: Status 404 returned error can't find the container with id af170d023a0bf46d4197afb3bf1fd0d18415225ff0c267cb6aab4ec371584a2c Jul 29 15:01:17.855964 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.855935 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-npfvr" Jul 29 15:01:17.860192 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:17.860171 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30d268aa_dda6_492b_acd6_b14a838c5948.slice/crio-b2f8d2856c5f094c830044097707ff1a3dd5a92b85a61a977b8d34c4c3e42617 WatchSource:0}: Error finding container b2f8d2856c5f094c830044097707ff1a3dd5a92b85a61a977b8d34c4c3e42617: Status 404 returned error can't find the container with id b2f8d2856c5f094c830044097707ff1a3dd5a92b85a61a977b8d34c4c3e42617 Jul 29 15:01:17.860858 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.860841 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:17.866059 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:17.866035 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6800b94b_4d3d_4b5a_ab8b_68083bbd2520.slice/crio-cf5c314823c97c635eb4f83496a9664aea1b15ec0d1baa94a9f74326cbcdc484 WatchSource:0}: Error finding container cf5c314823c97c635eb4f83496a9664aea1b15ec0d1baa94a9f74326cbcdc484: Status 404 returned error can't find the container with id cf5c314823c97c635eb4f83496a9664aea1b15ec0d1baa94a9f74326cbcdc484 Jul 29 15:01:17.868013 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.867977 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" Jul 29 15:01:17.875485 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:17.875465 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c52f73_7c5d_4452_96bc_4217e5341a27.slice/crio-b652d3bdd03bcf53768473217e9f45613d00147f816346a4be791a2d2e26b4c0 WatchSource:0}: Error finding container b652d3bdd03bcf53768473217e9f45613d00147f816346a4be791a2d2e26b4c0: Status 404 returned error can't find the container with id b652d3bdd03bcf53768473217e9f45613d00147f816346a4be791a2d2e26b4c0 Jul 29 15:01:17.876095 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:17.876081 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" Jul 29 15:01:17.883445 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:17.883417 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a1ce69_6788_4461_a3f8_e36f039aa692.slice/crio-42b13598980f013576a1b24a3b3986deda16430105de8e2d1b1731bdc8d9c601 WatchSource:0}: Error finding container 42b13598980f013576a1b24a3b3986deda16430105de8e2d1b1731bdc8d9c601: Status 404 returned error can't find the container with id 42b13598980f013576a1b24a3b3986deda16430105de8e2d1b1731bdc8d9c601 Jul 29 15:01:18.170109 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.170004 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:18.170109 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.170059 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:18.170109 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.170094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rc6\" (UniqueName: \"kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6\") pod \"network-check-target-2wrlr\" (UID: \"eeb6b594-9641-4f08-875d-6c6d1eb68cb1\") " pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:18.170354 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:18.170204 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:18.170354 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:18.170217 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:18.170354 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:18.170210 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 15:01:18.170354 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:18.170297 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 15:01:18.170354 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:18.170312 2562 projected.go:196] Error preparing data for projected volume kube-api-access-n6rc6 for pod openshift-network-diagnostics/network-check-target-2wrlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:18.170354 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:18.170277 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs podName:6c883406-1472-4688-819f-462bbb78df59 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:19.170259116 +0000 UTC m=+3.079107481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs") pod "network-metrics-daemon-v4svt" (UID: "6c883406-1472-4688-819f-462bbb78df59") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:18.170604 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:18.170379 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret podName:e7b52735-d789-4074-9c97-2e5bdb782caa nodeName:}" failed. No retries permitted until 2026-07-29 15:01:19.170362084 +0000 UTC m=+3.079210450 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret") pod "global-pull-secret-syncer-qc8sd" (UID: "e7b52735-d789-4074-9c97-2e5bdb782caa") : object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:18.170604 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:18.170402 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6 podName:eeb6b594-9641-4f08-875d-6c6d1eb68cb1 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:19.170391805 +0000 UTC m=+3.079240173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6rc6" (UniqueName: "kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6") pod "network-check-target-2wrlr" (UID: "eeb6b594-9641-4f08-875d-6c6d1eb68cb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:18.208018 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:18.207981 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19a4e2c03ee49097a3c2455ba7ff61f.slice/crio-7c3806bb3236f6d255bef37213f35a3751dc052323de91c60148b4df9f97de21 WatchSource:0}: Error finding container 7c3806bb3236f6d255bef37213f35a3751dc052323de91c60148b4df9f97de21: Status 404 returned error can't find the container with id 7c3806bb3236f6d255bef37213f35a3751dc052323de91c60148b4df9f97de21 Jul 29 15:01:18.238030 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.238002 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 15:01:18.514735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.514345 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 14:56:17 +0000 UTC" deadline="2028-01-10 10:54:17.808107962 +0000 UTC" Jul 29 15:01:18.514735 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.514413 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12715h52m59.293699144s" Jul 29 15:01:18.565908 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.565872 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:18.566084 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:18.566029 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:18.577982 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.577926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pp8jk" event={"ID":"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13","Type":"ContainerStarted","Data":"af170d023a0bf46d4197afb3bf1fd0d18415225ff0c267cb6aab4ec371584a2c"} Jul 29 15:01:18.585697 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.585663 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" event={"ID":"e3302ec9-0215-4448-b3c0-9b93ba589d31","Type":"ContainerStarted","Data":"bd5e2a2d26f38d278938d61149339fa7ab8914a6222b94d40fd7872a8e2cc842"} Jul 29 15:01:18.587947 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.587865 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-87r9f" event={"ID":"fd305d57-301d-47e9-87cb-f664d2a9c6f1","Type":"ContainerStarted","Data":"8c540c14e6dd399c87097185c3850770c9bd5c9bc0d72d0e1e54d7b3a802b0a6"} Jul 29 15:01:18.590274 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.590194 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" event={"ID":"c9a1ce69-6788-4461-a3f8-e36f039aa692","Type":"ContainerStarted","Data":"42b13598980f013576a1b24a3b3986deda16430105de8e2d1b1731bdc8d9c601"} Jul 29 15:01:18.595717 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.595652 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2brl7" event={"ID":"22bad190-4a5f-47f7-a261-25ec14e59621","Type":"ContainerStarted","Data":"be8a2f58388aef75a1ddad03bde2bffcc930f317bc59559bf6c6ff0613792273"} Jul 29 15:01:18.600529 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.600504 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-npfvr" event={"ID":"6800b94b-4d3d-4b5a-ab8b-68083bbd2520","Type":"ContainerStarted","Data":"cf5c314823c97c635eb4f83496a9664aea1b15ec0d1baa94a9f74326cbcdc484"} Jul 29 15:01:18.608217 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.608080 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" event={"ID":"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3","Type":"ContainerStarted","Data":"44d0e1b78aeffd0e5ae6f74dadfbf44fb804a78d30799b48fd3df574dd112586"} Jul 29 15:01:18.612185 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.612070 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal" event={"ID":"c19a4e2c03ee49097a3c2455ba7ff61f","Type":"ContainerStarted","Data":"7c3806bb3236f6d255bef37213f35a3751dc052323de91c60148b4df9f97de21"} Jul 29 15:01:18.618918 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.618834 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" event={"ID":"24c52f73-7c5d-4452-96bc-4217e5341a27","Type":"ContainerStarted","Data":"b652d3bdd03bcf53768473217e9f45613d00147f816346a4be791a2d2e26b4c0"} Jul 29 15:01:18.630572 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:18.630507 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rthsh" event={"ID":"30d268aa-dda6-492b-acd6-b14a838c5948","Type":"ContainerStarted","Data":"b2f8d2856c5f094c830044097707ff1a3dd5a92b85a61a977b8d34c4c3e42617"} Jul 29 15:01:19.180055 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:19.180017 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:19.180272 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:19.180087 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rc6\" (UniqueName: \"kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6\") pod \"network-check-target-2wrlr\" (UID: \"eeb6b594-9641-4f08-875d-6c6d1eb68cb1\") " pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:19.180272 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:19.180142 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:19.180396 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:19.180299 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:19.180396 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:19.180370 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret podName:e7b52735-d789-4074-9c97-2e5bdb782caa nodeName:}" failed. No retries permitted until 2026-07-29 15:01:21.180347325 +0000 UTC m=+5.089195711 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret") pod "global-pull-secret-syncer-qc8sd" (UID: "e7b52735-d789-4074-9c97-2e5bdb782caa") : object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:19.180844 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:19.180819 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:19.180922 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:19.180881 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs podName:6c883406-1472-4688-819f-462bbb78df59 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:21.180864669 +0000 UTC m=+5.089713044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs") pod "network-metrics-daemon-v4svt" (UID: "6c883406-1472-4688-819f-462bbb78df59") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:19.180993 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:19.180971 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 15:01:19.181050 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:19.181015 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 15:01:19.181050 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:19.181030 2562 projected.go:196] Error preparing data for projected volume kube-api-access-n6rc6 for pod openshift-network-diagnostics/network-check-target-2wrlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:19.181156 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:19.181076 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6 podName:eeb6b594-9641-4f08-875d-6c6d1eb68cb1 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:21.181061793 +0000 UTC m=+5.089910181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6rc6" (UniqueName: "kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6") pod "network-check-target-2wrlr" (UID: "eeb6b594-9641-4f08-875d-6c6d1eb68cb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:19.515562 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:19.515479 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 14:56:17 +0000 UTC" deadline="2028-05-04 18:41:51.29208724 +0000 UTC" Jul 29 15:01:19.515562 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:19.515521 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15483h40m31.776571393s" Jul 29 15:01:19.534306 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:19.534043 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 15:01:19.563706 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:19.563307 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:19.563706 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:19.563446 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:19.563706 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:19.563520 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:19.563706 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:19.563592 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:20.566225 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:20.565960 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:20.566623 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:20.566333 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:21.200985 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:21.200880 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:21.200985 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:21.200937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:21.201223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:21.201037 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rc6\" (UniqueName: \"kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6\") pod \"network-check-target-2wrlr\" (UID: \"eeb6b594-9641-4f08-875d-6c6d1eb68cb1\") " pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:21.201288 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:21.201216 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 15:01:21.201288 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:21.201229 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:21.201388 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:21.201301 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:21.201388 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:21.201316 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs podName:6c883406-1472-4688-819f-462bbb78df59 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:25.201291158 +0000 UTC m=+9.110139548 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs") pod "network-metrics-daemon-v4svt" (UID: "6c883406-1472-4688-819f-462bbb78df59") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:21.201388 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:21.201356 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret podName:e7b52735-d789-4074-9c97-2e5bdb782caa nodeName:}" failed. No retries permitted until 2026-07-29 15:01:25.201339558 +0000 UTC m=+9.110187940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret") pod "global-pull-secret-syncer-qc8sd" (UID: "e7b52735-d789-4074-9c97-2e5bdb782caa") : object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:21.201388 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:21.201243 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 15:01:21.201388 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:21.201382 2562 projected.go:196] Error preparing data for projected volume kube-api-access-n6rc6 for pod openshift-network-diagnostics/network-check-target-2wrlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:21.201658 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:21.201417 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6 podName:eeb6b594-9641-4f08-875d-6c6d1eb68cb1 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:25.201405465 +0000 UTC m=+9.110253854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6rc6" (UniqueName: "kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6") pod "network-check-target-2wrlr" (UID: "eeb6b594-9641-4f08-875d-6c6d1eb68cb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:21.564850 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:21.564153 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:21.564850 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:21.564295 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:21.564850 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:21.564375 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:21.564850 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:21.564472 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:21.650462 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:21.649644 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" event={"ID":"b336e36e39f016f7f0e11ef816392a81","Type":"ContainerStarted","Data":"38795d7ba4f5c5bd2f25d6ff0927092dd7999bd55afcd6ab33f7f6a55197cd53"} Jul 29 15:01:22.565693 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:22.564794 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:22.565895 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:22.565801 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:23.563404 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:23.563363 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:23.563903 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:23.563492 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:23.563903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:23.563569 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:23.563903 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:23.563716 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:24.563441 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:24.563402 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:24.563929 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:24.563540 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:25.238130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:25.238203 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:25.238253 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rc6\" (UniqueName: \"kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6\") pod \"network-check-target-2wrlr\" (UID: \"eeb6b594-9641-4f08-875d-6c6d1eb68cb1\") " pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:25.238264 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:25.238342 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret podName:e7b52735-d789-4074-9c97-2e5bdb782caa nodeName:}" failed. No retries permitted until 2026-07-29 15:01:33.238322848 +0000 UTC m=+17.147171206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret") pod "global-pull-secret-syncer-qc8sd" (UID: "e7b52735-d789-4074-9c97-2e5bdb782caa") : object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:25.238405 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:25.238429 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:25.238441 2562 projected.go:196] Error preparing data for projected volume kube-api-access-n6rc6 for pod openshift-network-diagnostics/network-check-target-2wrlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:25.238490 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6 podName:eeb6b594-9641-4f08-875d-6c6d1eb68cb1 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:33.238475429 +0000 UTC m=+17.147323805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6rc6" (UniqueName: "kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6") pod "network-check-target-2wrlr" (UID: "eeb6b594-9641-4f08-875d-6c6d1eb68cb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:25.238554 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:25.238638 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:25.238587 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs podName:6c883406-1472-4688-819f-462bbb78df59 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:33.238575684 +0000 UTC m=+17.147424044 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs") pod "network-metrics-daemon-v4svt" (UID: "6c883406-1472-4688-819f-462bbb78df59") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:25.563661 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:25.563628 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:25.564055 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:25.563766 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:25.564132 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:25.564117 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:25.564236 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:25.564213 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:26.565771 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:26.565738 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:26.566218 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:26.565842 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:27.564176 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:27.564138 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:27.564176 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:27.564170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:27.564414 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:27.564300 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:27.564461 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:27.564429 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:27.664277 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:27.664242 2562 generic.go:356] "Generic (PLEG): container finished" podID="b336e36e39f016f7f0e11ef816392a81" containerID="38795d7ba4f5c5bd2f25d6ff0927092dd7999bd55afcd6ab33f7f6a55197cd53" exitCode=0 Jul 29 15:01:27.664687 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:27.664289 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" event={"ID":"b336e36e39f016f7f0e11ef816392a81","Type":"ContainerDied","Data":"38795d7ba4f5c5bd2f25d6ff0927092dd7999bd55afcd6ab33f7f6a55197cd53"} Jul 29 15:01:28.563234 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:28.563203 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:28.563394 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:28.563312 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:29.563138 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:29.563107 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:29.563541 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:29.563256 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:29.563541 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:29.563300 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:29.563541 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:29.563414 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:30.563525 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:30.563441 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:30.564053 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:30.563575 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:31.564021 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:31.563992 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:31.564490 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:31.564114 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:31.564490 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:31.564180 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:31.564490 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:31.564275 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:32.564263 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:32.564229 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:32.564714 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:32.564362 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:33.301509 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:33.301476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:33.301743 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:33.301521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rc6\" (UniqueName: \"kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6\") pod \"network-check-target-2wrlr\" (UID: \"eeb6b594-9641-4f08-875d-6c6d1eb68cb1\") " pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:33.301743 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:33.301563 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:33.301743 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:33.301673 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:33.301743 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:33.301688 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:33.301743 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:33.301708 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 15:01:33.301743 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:33.301724 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 15:01:33.301743 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:33.301736 2562 projected.go:196] Error preparing data for projected volume kube-api-access-n6rc6 for pod openshift-network-diagnostics/network-check-target-2wrlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:33.302007 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:33.301753 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs podName:6c883406-1472-4688-819f-462bbb78df59 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:49.301731587 +0000 UTC m=+33.210579949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs") pod "network-metrics-daemon-v4svt" (UID: "6c883406-1472-4688-819f-462bbb78df59") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:33.302007 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:33.301775 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6 podName:eeb6b594-9641-4f08-875d-6c6d1eb68cb1 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:49.301761062 +0000 UTC m=+33.210609425 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6rc6" (UniqueName: "kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6") pod "network-check-target-2wrlr" (UID: "eeb6b594-9641-4f08-875d-6c6d1eb68cb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:33.302007 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:33.301793 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret podName:e7b52735-d789-4074-9c97-2e5bdb782caa nodeName:}" failed. No retries permitted until 2026-07-29 15:01:49.301784378 +0000 UTC m=+33.210632744 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret") pod "global-pull-secret-syncer-qc8sd" (UID: "e7b52735-d789-4074-9c97-2e5bdb782caa") : object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:33.564141 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:33.564057 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:33.564307 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:33.564202 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:33.564307 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:33.564263 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:33.564686 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:33.564404 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:34.566099 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:34.566021 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:34.566481 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:34.566133 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:35.563254 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:35.563189 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:35.563254 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:35.563255 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:35.563492 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:35.563345 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:35.563492 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:35.563463 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:36.564679 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:36.564639 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:36.565226 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:36.564755 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:37.563459 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:37.563427 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:37.563459 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:37.563457 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:37.563724 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:37.563557 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:37.563787 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:37.563719 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:38.567614 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:38.567518 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:38.568123 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:38.567641 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:39.563982 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.563611 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:39.564210 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.563671 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:39.564210 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:39.564101 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:39.564210 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:39.564166 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:39.685440 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.685360 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" event={"ID":"b336e36e39f016f7f0e11ef816392a81","Type":"ContainerStarted","Data":"64e25de6c6971e1a9920a21ed90b1b002d1d3128bb5b1550f5fca07ee6bae455"} Jul 29 15:01:39.686970 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.686946 2562 generic.go:356] "Generic (PLEG): container finished" podID="c9a1ce69-6788-4461-a3f8-e36f039aa692" containerID="38c04d2bd6859b6b00a861b3968732d1c7f0a1b58d8b4f17a1d22945779fd2e7" exitCode=0 Jul 29 15:01:39.687077 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.687011 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" event={"ID":"c9a1ce69-6788-4461-a3f8-e36f039aa692","Type":"ContainerDied","Data":"38c04d2bd6859b6b00a861b3968732d1c7f0a1b58d8b4f17a1d22945779fd2e7"} Jul 29 15:01:39.688523 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.688441 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2brl7" event={"ID":"22bad190-4a5f-47f7-a261-25ec14e59621","Type":"ContainerStarted","Data":"80f529644f44310248b7229dd3a42697f260aabc9d5e85a5f7d182729831cbb9"} Jul 29 15:01:39.690061 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.690005 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-npfvr" event={"ID":"6800b94b-4d3d-4b5a-ab8b-68083bbd2520","Type":"ContainerStarted","Data":"3ff70d6fc6f7c8aa16a0f8ad285a9360cce2311d61777939f88a1e0275cb82f3"} Jul 29 15:01:39.692868 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.692849 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" event={"ID":"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3","Type":"ContainerStarted","Data":"5dd52940ac06c01fb842d7d176f25c4b87898777a05a86cbea436f80860c5635"} Jul 29 15:01:39.692981 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.692920 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" event={"ID":"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3","Type":"ContainerStarted","Data":"74a01cfcc3fb93c03ed39dc1fad134be3bc472d8b22f8b520f42cd6fe3ef132d"} Jul 29 15:01:39.692981 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.692974 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" event={"ID":"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3","Type":"ContainerStarted","Data":"04d83c516450354c7070946bd700b8fb2b536b3aed764a89d9f6e83d9dd69b00"} Jul 29 15:01:39.693091 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.693021 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" event={"ID":"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3","Type":"ContainerStarted","Data":"bd33bb5f0c1b64c2c7f2ec143abd3442921bf50c585d7383bdd8cf3399235313"} Jul 29 15:01:39.693091 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.693068 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" event={"ID":"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3","Type":"ContainerStarted","Data":"5fbaf3ed2434e1905d2e9acddd17ac892a4ee84bc97cee0c58a30354770bdd74"} Jul 29 15:01:39.693182 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.693115 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" event={"ID":"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3","Type":"ContainerStarted","Data":"902d98124f6edbc5cad90cf7f138ee5f546c6dcf63b980bee248811c0e9a7c29"} Jul 29 15:01:39.694390 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.694367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal" event={"ID":"c19a4e2c03ee49097a3c2455ba7ff61f","Type":"ContainerStarted","Data":"c9847d49703c9ed999fe87c5b38d02daedfaa88f4887f8ee1b38c6f305056afe"} Jul 29 15:01:39.695720 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.695701 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" event={"ID":"24c52f73-7c5d-4452-96bc-4217e5341a27","Type":"ContainerStarted","Data":"41fe6b018de0bc93327a304291e9b0f587a6108ada70d0e51a7cabfea5251b09"} Jul 29 15:01:39.697025 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.697006 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rthsh" event={"ID":"30d268aa-dda6-492b-acd6-b14a838c5948","Type":"ContainerStarted","Data":"be6177bf6f39a0a938d000bf20284d9fdaffe8a76cf32c698bcf4ae9fba84915"} Jul 29 15:01:39.698449 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.698426 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pp8jk" event={"ID":"c20ebebb-b1b8-4d52-b617-b6faa3d4bf13","Type":"ContainerStarted","Data":"f4f2a1020ada0a8a80baaebb9d9431253eeede8bbc40be8a36b67e42b0eb239e"} Jul 29 15:01:39.699770 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.699753 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" event={"ID":"e3302ec9-0215-4448-b3c0-9b93ba589d31","Type":"ContainerStarted","Data":"23e42736f3e955e69a027d963d3c92bc481e1b4bb379d44aec1e979dc2214926"} Jul 29 15:01:39.703043 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.703007 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-104.ec2.internal" podStartSLOduration=22.702995962 podStartE2EDuration="22.702995962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:39.70263055 +0000 UTC m=+23.611478925" watchObservedRunningTime="2026-07-29 15:01:39.702995962 +0000 UTC m=+23.611844342" Jul 29 15:01:39.718467 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.718417 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-104.ec2.internal" podStartSLOduration=22.718403568 podStartE2EDuration="22.718403568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:39.717885211 +0000 UTC m=+23.626733593" watchObservedRunningTime="2026-07-29 15:01:39.718403568 +0000 UTC m=+23.627251948" Jul 29 15:01:39.735391 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.735302 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pp8jk" podStartSLOduration=1.912068949 podStartE2EDuration="22.735288166s" totalImagesPullingTime="20.823219217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:39.734923642 +0000 UTC m=+23.643772022" watchObservedRunningTime="2026-07-29 15:01:39.735288166 +0000 UTC m=+23.644136547" Jul 29 15:01:39.783935 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.783883 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rthsh" podStartSLOduration=1.983675005 podStartE2EDuration="22.783862703s" totalImagesPullingTime="20.800187698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:39.765670782 +0000 UTC m=+23.674519163" watchObservedRunningTime="2026-07-29 15:01:39.783862703 +0000 UTC m=+23.692711086" Jul 29 15:01:39.784096 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.783954 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-npfvr" podStartSLOduration=1.936654173 podStartE2EDuration="22.78395068s" totalImagesPullingTime="20.847296507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:39.783494544 +0000 UTC m=+23.692342925" watchObservedRunningTime="2026-07-29 15:01:39.78395068 +0000 UTC m=+23.692799061" Jul 29 15:01:39.801538 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.801491 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2brl7" podStartSLOduration=1.9900627709999998 podStartE2EDuration="22.801473243s" totalImagesPullingTime="20.811410472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:39.800922685 +0000 UTC m=+23.709771113" watchObservedRunningTime="2026-07-29 15:01:39.801473243 +0000 UTC m=+23.710321624" Jul 29 15:01:39.829998 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:39.829956 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lzwd6" podStartSLOduration=2.048007269 podStartE2EDuration="22.829941808s" totalImagesPullingTime="20.781934539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:39.829637314 +0000 UTC m=+23.738485692" watchObservedRunningTime="2026-07-29 15:01:39.829941808 +0000 UTC m=+23.738790189" Jul 29 15:01:40.259709 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:40.259688 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 29 15:01:40.532310 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:40.532219 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T15:01:40.259704831Z","UUID":"972534ad-cdbf-41eb-8819-e81ae08c179d","Handler":null,"Name":"","Endpoint":""} Jul 29 15:01:40.534154 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:40.534126 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 29 15:01:40.534286 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:40.534171 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 29 15:01:40.564198 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:40.564168 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:40.564420 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:40.564287 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:40.704151 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:40.704113 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" event={"ID":"e3302ec9-0215-4448-b3c0-9b93ba589d31","Type":"ContainerStarted","Data":"b40b077ca9afb2271cce5369f0cad65c4d6ccb2b5084eab17aaee4d9d8544e42"} Jul 29 15:01:40.705776 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:40.705746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-87r9f" event={"ID":"fd305d57-301d-47e9-87cb-f664d2a9c6f1","Type":"ContainerStarted","Data":"aa75d7df48da503a1530167e106a4363eb655defb68499d92ae261184956bc23"} Jul 29 15:01:40.725435 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:40.725384 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-87r9f" podStartSLOduration=2.860928818 podStartE2EDuration="23.725363704s" totalImagesPullingTime="20.864434886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:40.725190454 +0000 UTC m=+24.634038837" watchObservedRunningTime="2026-07-29 15:01:40.725363704 +0000 UTC m=+24.634212087" Jul 29 15:01:41.563214 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:41.563186 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:41.563342 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:41.563299 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:41.563406 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:41.563366 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:41.563534 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:41.563503 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:41.710767 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:41.710687 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" event={"ID":"e3302ec9-0215-4448-b3c0-9b93ba589d31","Type":"ContainerStarted","Data":"5cc3cf617291e5d3d8527e213fcf4b13bc6278236a84c19befcfa102c7dfd204"} Jul 29 15:01:41.715643 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:41.715618 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" event={"ID":"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3","Type":"ContainerStarted","Data":"5bede08cb301a5680dec541f83e9ae46734f5f9a1b91aca9486b0b1aa4b45531"} Jul 29 15:01:41.729655 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:41.729552 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcwvp" podStartSLOduration=1.396219229 podStartE2EDuration="24.72953104s" totalImagesPullingTime="23.333311811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:41.729136712 +0000 UTC m=+25.637985093" watchObservedRunningTime="2026-07-29 15:01:41.72953104 +0000 UTC m=+25.638379416" Jul 29 15:01:42.127522 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:42.127301 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:42.128774 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:42.128747 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:42.563279 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:42.563242 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:42.563459 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:42.563409 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:42.719005 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:42.718975 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:42.719519 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:42.719499 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2brl7" Jul 29 15:01:43.563674 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:43.563639 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:43.563963 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:43.563639 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:43.563963 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:43.563779 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:43.563963 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:43.563829 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:44.563666 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:44.563474 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:44.564739 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:44.563746 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:44.725253 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:44.725168 2562 generic.go:356] "Generic (PLEG): container finished" podID="c9a1ce69-6788-4461-a3f8-e36f039aa692" containerID="4aba78968bfba2f8baffb3960d1447b13603e5ec8d3fc210aeeceffd91fe26df" exitCode=0 Jul 29 15:01:44.725394 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:44.725247 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" event={"ID":"c9a1ce69-6788-4461-a3f8-e36f039aa692","Type":"ContainerDied","Data":"4aba78968bfba2f8baffb3960d1447b13603e5ec8d3fc210aeeceffd91fe26df"} Jul 29 15:01:44.729019 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:44.728991 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" event={"ID":"4e087d29-d6fa-400e-a408-7d0a2cf8b6f3","Type":"ContainerStarted","Data":"39754da6ae3629fc0cce3db49fa0624b964668174688aa30965cbdaf6d05e8d2"} Jul 29 15:01:44.729274 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:44.729240 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:44.729384 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:44.729302 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:44.745441 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:44.745411 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:44.818904 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:44.818849 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" podStartSLOduration=6.864995859 podStartE2EDuration="27.818834185s" totalImagesPullingTime="20.953838326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:44.817175989 +0000 UTC m=+28.726024382" watchObservedRunningTime="2026-07-29 15:01:44.818834185 +0000 UTC m=+28.727682565" Jul 29 15:01:45.563477 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:45.563440 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:45.563477 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:45.563471 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:45.563730 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:45.563561 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:45.563730 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:45.563689 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:45.732406 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:45.732366 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:45.747988 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:45.747961 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:01:46.429575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:46.427665 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v4svt"] Jul 29 15:01:46.429575 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:46.427848 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:46.429575 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:46.427996 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:46.430869 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:46.430839 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2wrlr"] Jul 29 15:01:46.430993 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:46.430960 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:46.431071 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:46.431049 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:46.431454 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:46.431426 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qc8sd"] Jul 29 15:01:46.431525 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:46.431513 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:46.431623 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:46.431586 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:46.736394 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:46.736316 2562 generic.go:356] "Generic (PLEG): container finished" podID="c9a1ce69-6788-4461-a3f8-e36f039aa692" containerID="f83a95b4ba18ad363d77dd1939d76d801f9dbc297405c5863d29f6836e489200" exitCode=0 Jul 29 15:01:46.736887 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:46.736406 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" event={"ID":"c9a1ce69-6788-4461-a3f8-e36f039aa692","Type":"ContainerDied","Data":"f83a95b4ba18ad363d77dd1939d76d801f9dbc297405c5863d29f6836e489200"} Jul 29 15:01:47.563668 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:47.563635 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:47.563829 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:47.563739 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:48.564456 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:48.564198 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:48.565084 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:48.564706 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:48.565084 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:48.564765 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:48.565330 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:48.565291 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:48.760118 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:48.760081 2562 generic.go:356] "Generic (PLEG): container finished" podID="c9a1ce69-6788-4461-a3f8-e36f039aa692" containerID="e54c38cfb00992d74e473c0ebe87f80126827843672411d343ac77d1a65336b6" exitCode=0 Jul 29 15:01:48.760258 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:48.760136 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" event={"ID":"c9a1ce69-6788-4461-a3f8-e36f039aa692","Type":"ContainerDied","Data":"e54c38cfb00992d74e473c0ebe87f80126827843672411d343ac77d1a65336b6"} Jul 29 15:01:49.326444 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:49.326400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:49.326444 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:49.326450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rc6\" (UniqueName: \"kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6\") pod \"network-check-target-2wrlr\" (UID: \"eeb6b594-9641-4f08-875d-6c6d1eb68cb1\") " pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:49.326684 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:49.326492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:49.326684 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:49.326590 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:49.326684 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:49.326616 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:49.326684 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:49.326679 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs podName:6c883406-1472-4688-819f-462bbb78df59 nodeName:}" failed. No retries permitted until 2026-07-29 15:02:21.326650345 +0000 UTC m=+65.235498705 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs") pod "network-metrics-daemon-v4svt" (UID: "6c883406-1472-4688-819f-462bbb78df59") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 15:01:49.326840 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:49.326692 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret podName:e7b52735-d789-4074-9c97-2e5bdb782caa nodeName:}" failed. No retries permitted until 2026-07-29 15:02:21.326686346 +0000 UTC m=+65.235534705 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret") pod "global-pull-secret-syncer-qc8sd" (UID: "e7b52735-d789-4074-9c97-2e5bdb782caa") : object "kube-system"/"original-pull-secret" not registered Jul 29 15:01:49.326840 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:49.326763 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 15:01:49.326840 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:49.326787 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 15:01:49.326840 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:49.326802 2562 projected.go:196] Error preparing data for projected volume kube-api-access-n6rc6 for pod openshift-network-diagnostics/network-check-target-2wrlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:49.326980 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:49.326891 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6 podName:eeb6b594-9641-4f08-875d-6c6d1eb68cb1 nodeName:}" failed. No retries permitted until 2026-07-29 15:02:21.326874356 +0000 UTC m=+65.235722718 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6rc6" (UniqueName: "kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6") pod "network-check-target-2wrlr" (UID: "eeb6b594-9641-4f08-875d-6c6d1eb68cb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 15:01:49.563837 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:49.563802 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:49.564028 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:49.563929 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wrlr" podUID="eeb6b594-9641-4f08-875d-6c6d1eb68cb1" Jul 29 15:01:50.563249 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:50.563212 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:50.563667 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:50.563364 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:01:50.563667 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:50.563212 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:50.563667 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:50.563466 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qc8sd" podUID="e7b52735-d789-4074-9c97-2e5bdb782caa" Jul 29 15:01:51.485292 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.485211 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-104.ec2.internal" event="NodeReady" Jul 29 15:01:51.485433 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.485349 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 29 15:01:51.521180 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.521145 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-746cbbbd85-5hv56"] Jul 29 15:01:51.523233 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.523209 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.525672 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.525632 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 29 15:01:51.525672 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.525659 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tlvgb\"" Jul 29 15:01:51.525892 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.525873 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 29 15:01:51.526014 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.525919 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 29 15:01:51.533500 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.533478 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 29 15:01:51.537833 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.537810 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-746cbbbd85-5hv56"] Jul 29 15:01:51.541320 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.541296 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hv4wr"] Jul 29 15:01:51.543302 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.543284 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hv4wr" Jul 29 15:01:51.546520 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.546487 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 29 15:01:51.546644 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.546625 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q9b9z\"" Jul 29 15:01:51.546788 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.546766 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 29 15:01:51.547552 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.547533 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 29 15:01:51.553859 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.553836 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hv4wr"] Jul 29 15:01:51.564023 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.563999 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:01:51.567988 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.567970 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cvznn\"" Jul 29 15:01:51.568108 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.568082 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 15:01:51.574321 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.574297 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 15:01:51.635323 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.635287 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7dqs7"] Jul 29 15:01:51.637271 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.637245 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.640206 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.640186 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 29 15:01:51.640622 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.640587 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w95m5\"" Jul 29 15:01:51.640734 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.640632 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 29 15:01:51.643565 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.643540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-image-registry-private-configuration\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.643694 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.643571 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-trusted-ca\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.643694 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.643588 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pml9\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-kube-api-access-9pml9\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.643694 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.643632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsfkd\" (UniqueName: \"kubernetes.io/projected/5318ca26-8284-482e-ab92-a3cb709fb40c-kube-api-access-bsfkd\") pod \"ingress-canary-hv4wr\" (UID: \"5318ca26-8284-482e-ab92-a3cb709fb40c\") " pod="openshift-ingress-canary/ingress-canary-hv4wr" Jul 29 15:01:51.643835 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.643705 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/716f69de-0819-40aa-90f4-db4954f5bbcf-ca-trust-extracted\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.643835 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.643732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-certificates\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.643835 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.643781 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.643965 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.643847 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5318ca26-8284-482e-ab92-a3cb709fb40c-cert\") pod \"ingress-canary-hv4wr\" (UID: \"5318ca26-8284-482e-ab92-a3cb709fb40c\") " pod="openshift-ingress-canary/ingress-canary-hv4wr" Jul 29 15:01:51.643965 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.643868 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-installation-pull-secrets\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.643965 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.643898 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-bound-sa-token\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.653361 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.653338 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7dqs7"] Jul 29 15:01:51.745247 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745210 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/716f69de-0819-40aa-90f4-db4954f5bbcf-ca-trust-extracted\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.745399 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-certificates\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.745399 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745326 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.745399 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfbfaa07-7732-4510-bfad-041401ba0c43-tmp-dir\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.745399 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745368 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfbfaa07-7732-4510-bfad-041401ba0c43-config-volume\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.745672 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745403 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5318ca26-8284-482e-ab92-a3cb709fb40c-cert\") pod \"ingress-canary-hv4wr\" (UID: \"5318ca26-8284-482e-ab92-a3cb709fb40c\") " pod="openshift-ingress-canary/ingress-canary-hv4wr" Jul 29 15:01:51.745672 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-installation-pull-secrets\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.745672 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-bound-sa-token\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.745672 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:51.745453 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 15:01:51.745672 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:51.745474 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-746cbbbd85-5hv56: secret "image-registry-tls" not found Jul 29 15:01:51.745672 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-image-registry-private-configuration\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.745672 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:51.745540 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls podName:716f69de-0819-40aa-90f4-db4954f5bbcf nodeName:}" failed. No retries permitted until 2026-07-29 15:01:52.245518672 +0000 UTC m=+36.154367051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls") pod "image-registry-746cbbbd85-5hv56" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf") : secret "image-registry-tls" not found Jul 29 15:01:51.745672 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745576 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-trusted-ca\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.745672 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pml9\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-kube-api-access-9pml9\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.745672 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.746114 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745689 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsfkd\" (UniqueName: \"kubernetes.io/projected/5318ca26-8284-482e-ab92-a3cb709fb40c-kube-api-access-bsfkd\") pod \"ingress-canary-hv4wr\" (UID: \"5318ca26-8284-482e-ab92-a3cb709fb40c\") " pod="openshift-ingress-canary/ingress-canary-hv4wr" Jul 29 15:01:51.746114 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745714 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62lk5\" (UniqueName: \"kubernetes.io/projected/bfbfaa07-7732-4510-bfad-041401ba0c43-kube-api-access-62lk5\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.746114 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745716 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/716f69de-0819-40aa-90f4-db4954f5bbcf-ca-trust-extracted\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.746114 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.745900 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-certificates\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.746501 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.746480 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-trusted-ca\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.750029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.750008 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-installation-pull-secrets\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.750142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.750100 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-image-registry-private-configuration\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.750186 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.750150 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5318ca26-8284-482e-ab92-a3cb709fb40c-cert\") pod \"ingress-canary-hv4wr\" (UID: \"5318ca26-8284-482e-ab92-a3cb709fb40c\") " pod="openshift-ingress-canary/ingress-canary-hv4wr" Jul 29 15:01:51.755027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.755004 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-bound-sa-token\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.755138 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.755071 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsfkd\" (UniqueName: \"kubernetes.io/projected/5318ca26-8284-482e-ab92-a3cb709fb40c-kube-api-access-bsfkd\") pod \"ingress-canary-hv4wr\" (UID: \"5318ca26-8284-482e-ab92-a3cb709fb40c\") " pod="openshift-ingress-canary/ingress-canary-hv4wr" Jul 29 15:01:51.755272 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.755249 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pml9\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-kube-api-access-9pml9\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:51.767439 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.767356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:01:51.846143 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.846102 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.846143 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.846142 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62lk5\" (UniqueName: \"kubernetes.io/projected/bfbfaa07-7732-4510-bfad-041401ba0c43-kube-api-access-62lk5\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.846389 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.846192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfbfaa07-7732-4510-bfad-041401ba0c43-tmp-dir\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.846389 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.846224 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfbfaa07-7732-4510-bfad-041401ba0c43-config-volume\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.846389 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:51.846271 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 15:01:51.846389 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:51.846343 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls podName:bfbfaa07-7732-4510-bfad-041401ba0c43 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:52.3463238 +0000 UTC m=+36.255172189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls") pod "dns-default-7dqs7" (UID: "bfbfaa07-7732-4510-bfad-041401ba0c43") : secret "dns-default-metrics-tls" not found Jul 29 15:01:51.846624 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.846553 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfbfaa07-7732-4510-bfad-041401ba0c43-tmp-dir\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.846886 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.846865 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfbfaa07-7732-4510-bfad-041401ba0c43-config-volume\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.853237 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.853208 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hv4wr" Jul 29 15:01:51.856314 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.856296 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62lk5\" (UniqueName: \"kubernetes.io/projected/bfbfaa07-7732-4510-bfad-041401ba0c43-kube-api-access-62lk5\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:51.861373 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:51.861346 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:01:52.249058 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:52.249008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:52.249228 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:52.249181 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 15:01:52.249228 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:52.249203 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-746cbbbd85-5hv56: secret "image-registry-tls" not found Jul 29 15:01:52.249303 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:52.249260 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls podName:716f69de-0819-40aa-90f4-db4954f5bbcf nodeName:}" failed. No retries permitted until 2026-07-29 15:01:53.249244359 +0000 UTC m=+37.158092722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls") pod "image-registry-746cbbbd85-5hv56" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf") : secret "image-registry-tls" not found Jul 29 15:01:52.349944 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:52.349902 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:52.350127 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:52.350032 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 15:01:52.350127 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:52.350119 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls podName:bfbfaa07-7732-4510-bfad-041401ba0c43 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:53.350101201 +0000 UTC m=+37.258949559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls") pod "dns-default-7dqs7" (UID: "bfbfaa07-7732-4510-bfad-041401ba0c43") : secret "dns-default-metrics-tls" not found Jul 29 15:01:52.567334 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:52.567254 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:01:52.567334 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:52.567270 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:01:52.569745 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:52.569582 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g48bm\"" Jul 29 15:01:52.569745 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:52.569614 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 15:01:52.569745 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:52.569656 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 15:01:52.965855 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:52.965702 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hv4wr"] Jul 29 15:01:52.969456 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:01:52.969427 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5318ca26_8284_482e_ab92_a3cb709fb40c.slice/crio-399d50ff5e81a94ad46ef73d5a49f02cff8bfc3b575c85855a04915099922eeb WatchSource:0}: Error finding container 399d50ff5e81a94ad46ef73d5a49f02cff8bfc3b575c85855a04915099922eeb: Status 404 returned error can't find the container with id 399d50ff5e81a94ad46ef73d5a49f02cff8bfc3b575c85855a04915099922eeb Jul 29 15:01:53.256652 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:53.256612 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:53.256822 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:53.256741 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 15:01:53.256822 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:53.256754 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-746cbbbd85-5hv56: secret "image-registry-tls" not found Jul 29 15:01:53.256822 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:53.256803 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls podName:716f69de-0819-40aa-90f4-db4954f5bbcf nodeName:}" failed. No retries permitted until 2026-07-29 15:01:55.256789833 +0000 UTC m=+39.165638191 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls") pod "image-registry-746cbbbd85-5hv56" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf") : secret "image-registry-tls" not found Jul 29 15:01:53.357177 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:53.357083 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:53.357340 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:53.357265 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 15:01:53.357395 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:53.357342 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls podName:bfbfaa07-7732-4510-bfad-041401ba0c43 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:55.357321758 +0000 UTC m=+39.266170116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls") pod "dns-default-7dqs7" (UID: "bfbfaa07-7732-4510-bfad-041401ba0c43") : secret "dns-default-metrics-tls" not found Jul 29 15:01:53.774042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:53.774011 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hv4wr" event={"ID":"5318ca26-8284-482e-ab92-a3cb709fb40c","Type":"ContainerStarted","Data":"399d50ff5e81a94ad46ef73d5a49f02cff8bfc3b575c85855a04915099922eeb"} Jul 29 15:01:53.776434 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:53.776411 2562 generic.go:356] "Generic (PLEG): container finished" podID="c9a1ce69-6788-4461-a3f8-e36f039aa692" containerID="4d8036b6174d20c8db12c4ae65d091957775eb9d33d5c85359664499949b0aa9" exitCode=0 Jul 29 15:01:53.776546 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:53.776452 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" event={"ID":"c9a1ce69-6788-4461-a3f8-e36f039aa692","Type":"ContainerDied","Data":"4d8036b6174d20c8db12c4ae65d091957775eb9d33d5c85359664499949b0aa9"} Jul 29 15:01:54.630289 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:54.630238 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:01:54.726692 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:54.726652 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:01:54.782878 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:54.782840 2562 generic.go:356] "Generic (PLEG): container finished" podID="c9a1ce69-6788-4461-a3f8-e36f039aa692" containerID="4b2f312406e16f33ba71d18f6f212a601683105dc932a2ce478c3655cb7a3ea6" exitCode=0 Jul 29 15:01:54.783309 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:54.782891 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" event={"ID":"c9a1ce69-6788-4461-a3f8-e36f039aa692","Type":"ContainerDied","Data":"4b2f312406e16f33ba71d18f6f212a601683105dc932a2ce478c3655cb7a3ea6"} Jul 29 15:01:54.788949 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:54.788917 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:01:55.271739 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:55.271695 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:55.271931 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:55.271877 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 15:01:55.271931 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:55.271900 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-746cbbbd85-5hv56: secret "image-registry-tls" not found Jul 29 15:01:55.272041 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:55.271967 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls podName:716f69de-0819-40aa-90f4-db4954f5bbcf nodeName:}" failed. No retries permitted until 2026-07-29 15:01:59.271946319 +0000 UTC m=+43.180794679 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls") pod "image-registry-746cbbbd85-5hv56" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf") : secret "image-registry-tls" not found Jul 29 15:01:55.372226 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:55.372184 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:55.372424 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:55.372349 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 15:01:55.372478 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:55.372431 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls podName:bfbfaa07-7732-4510-bfad-041401ba0c43 nodeName:}" failed. No retries permitted until 2026-07-29 15:01:59.372407259 +0000 UTC m=+43.281255623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls") pod "dns-default-7dqs7" (UID: "bfbfaa07-7732-4510-bfad-041401ba0c43") : secret "dns-default-metrics-tls" not found Jul 29 15:01:55.569191 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:55.569005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:01:55.667634 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:55.667580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:01:55.787631 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:55.787569 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hv4wr" event={"ID":"5318ca26-8284-482e-ab92-a3cb709fb40c","Type":"ContainerStarted","Data":"b00de9335bdf1e54d546cbd0b8196b619499f339efe7a01574c45c0542175c52"} Jul 29 15:01:55.792340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:55.792306 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" event={"ID":"c9a1ce69-6788-4461-a3f8-e36f039aa692","Type":"ContainerStarted","Data":"0e3e2501bae40d22d8c057af3d3f6df20a284265b8d7a2876b450433196bb1c5"} Jul 29 15:01:55.804963 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:55.804849 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hv4wr" podStartSLOduration=2.274122737 podStartE2EDuration="4.8048332s" totalImagesPullingTime="2.530710463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:01:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:55.804313923 +0000 UTC m=+39.713162306" watchObservedRunningTime="2026-07-29 15:01:55.8048332 +0000 UTC m=+39.713681580" Jul 29 15:01:55.828000 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:55.827804 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ftrpv" podStartSLOduration=7.226842584 podStartE2EDuration="38.827783213s" totalImagesPullingTime="31.600940629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:01:55.827319239 +0000 UTC m=+39.736167657" watchObservedRunningTime="2026-07-29 15:01:55.827783213 +0000 UTC m=+39.736631595" Jul 29 15:01:55.949042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:55.949006 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:01:56.014772 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:56.014728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:01:59.301173 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:59.301124 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:01:59.301543 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:59.301238 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 15:01:59.301543 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:59.301259 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-746cbbbd85-5hv56: secret "image-registry-tls" not found Jul 29 15:01:59.301543 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:59.301317 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls podName:716f69de-0819-40aa-90f4-db4954f5bbcf nodeName:}" failed. No retries permitted until 2026-07-29 15:02:07.301300272 +0000 UTC m=+51.210148631 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls") pod "image-registry-746cbbbd85-5hv56" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf") : secret "image-registry-tls" not found Jul 29 15:01:59.401992 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:01:59.401953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:01:59.402180 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:59.402141 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 15:01:59.402245 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:01:59.402223 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls podName:bfbfaa07-7732-4510-bfad-041401ba0c43 nodeName:}" failed. No retries permitted until 2026-07-29 15:02:07.402202418 +0000 UTC m=+51.311050780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls") pod "dns-default-7dqs7" (UID: "bfbfaa07-7732-4510-bfad-041401ba0c43") : secret "dns-default-metrics-tls" not found Jul 29 15:02:01.126313 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:01.126284 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:01.228420 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:01.228383 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:06.851805 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:06.851773 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:06.956123 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:06.956071 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:07.052718 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:07.052683 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:07.154884 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:07.154806 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:07.254638 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:07.254584 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:07.367514 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:07.367469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:02:07.367707 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:07.367632 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 15:02:07.367707 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:07.367648 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-746cbbbd85-5hv56: secret "image-registry-tls" not found Jul 29 15:02:07.367799 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:07.367721 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls podName:716f69de-0819-40aa-90f4-db4954f5bbcf nodeName:}" failed. No retries permitted until 2026-07-29 15:02:23.367704389 +0000 UTC m=+67.276552747 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls") pod "image-registry-746cbbbd85-5hv56" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf") : secret "image-registry-tls" not found Jul 29 15:02:07.468580 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:07.468492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:02:07.468769 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:07.468677 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 15:02:07.468769 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:07.468765 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls podName:bfbfaa07-7732-4510-bfad-041401ba0c43 nodeName:}" failed. No retries permitted until 2026-07-29 15:02:23.468744587 +0000 UTC m=+67.377592952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls") pod "dns-default-7dqs7" (UID: "bfbfaa07-7732-4510-bfad-041401ba0c43") : secret "dns-default-metrics-tls" not found Jul 29 15:02:07.739534 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:07.739505 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:07.841234 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:07.841186 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:07.941366 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:07.941324 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:08.003155 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:08.003081 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:08.833912 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:08.833879 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:10.661340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:10.661301 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:10.761851 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:10.761812 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:11.272672 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:11.272637 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:11.374251 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:11.374216 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:13.868200 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:13.868161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:17.750463 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:17.750435 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmhbw" Jul 29 15:02:17.921029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:17.920995 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:17.995391 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:17.995355 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:20.032928 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:20.032887 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:20.131288 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:20.131251 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:21.367247 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.367207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:02:21.367247 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.367254 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rc6\" (UniqueName: \"kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6\") pod \"network-check-target-2wrlr\" (UID: \"eeb6b594-9641-4f08-875d-6c6d1eb68cb1\") " pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:02:21.367726 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.367294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:02:21.369418 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.369395 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 15:02:21.369488 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.369458 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 15:02:21.369539 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.369511 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 15:02:21.378408 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:21.378372 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 29 15:02:21.378552 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:21.378457 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs podName:6c883406-1472-4688-819f-462bbb78df59 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:25.378433717 +0000 UTC m=+129.287282076 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs") pod "network-metrics-daemon-v4svt" (UID: "6c883406-1472-4688-819f-462bbb78df59") : secret "metrics-daemon-secret" not found Jul 29 15:02:21.380380 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.380358 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 15:02:21.382492 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.382470 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7b52735-d789-4074-9c97-2e5bdb782caa-original-pull-secret\") pod \"global-pull-secret-syncer-qc8sd\" (UID: \"e7b52735-d789-4074-9c97-2e5bdb782caa\") " pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:02:21.390841 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.390818 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6rc6\" (UniqueName: \"kubernetes.io/projected/eeb6b594-9641-4f08-875d-6c6d1eb68cb1-kube-api-access-n6rc6\") pod \"network-check-target-2wrlr\" (UID: \"eeb6b594-9641-4f08-875d-6c6d1eb68cb1\") " pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:02:21.575905 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.575877 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cvznn\"" Jul 29 15:02:21.584270 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.584235 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:02:21.678792 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.678758 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qc8sd" Jul 29 15:02:21.739384 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.739357 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2wrlr"] Jul 29 15:02:21.742897 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:02:21.742861 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb6b594_9641_4f08_875d_6c6d1eb68cb1.slice/crio-f20cde93f4e4ddfa3f92d0d391cacceb81cc659983ce7744db758d3dbc6c3ba4 WatchSource:0}: Error finding container f20cde93f4e4ddfa3f92d0d391cacceb81cc659983ce7744db758d3dbc6c3ba4: Status 404 returned error can't find the container with id f20cde93f4e4ddfa3f92d0d391cacceb81cc659983ce7744db758d3dbc6c3ba4 Jul 29 15:02:21.802001 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.801968 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qc8sd"] Jul 29 15:02:21.805176 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:02:21.805156 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b52735_d789_4074_9c97_2e5bdb782caa.slice/crio-169f2ee30c8b4cd98a5796c122710ee1286eabfb70cf0cd63acda49974195370 WatchSource:0}: Error finding container 169f2ee30c8b4cd98a5796c122710ee1286eabfb70cf0cd63acda49974195370: Status 404 returned error can't find the container with id 169f2ee30c8b4cd98a5796c122710ee1286eabfb70cf0cd63acda49974195370 Jul 29 15:02:21.855743 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.855702 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qc8sd" event={"ID":"e7b52735-d789-4074-9c97-2e5bdb782caa","Type":"ContainerStarted","Data":"169f2ee30c8b4cd98a5796c122710ee1286eabfb70cf0cd63acda49974195370"} Jul 29 15:02:21.856774 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:21.856745 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2wrlr" event={"ID":"eeb6b594-9641-4f08-875d-6c6d1eb68cb1","Type":"ContainerStarted","Data":"f20cde93f4e4ddfa3f92d0d391cacceb81cc659983ce7744db758d3dbc6c3ba4"} Jul 29 15:02:22.972818 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:22.972749 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:23.054049 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:23.053953 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:23.380977 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:23.380944 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:02:23.381170 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:23.381083 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 15:02:23.381170 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:23.381099 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-746cbbbd85-5hv56: secret "image-registry-tls" not found Jul 29 15:02:23.381170 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:23.381161 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls podName:716f69de-0819-40aa-90f4-db4954f5bbcf nodeName:}" failed. No retries permitted until 2026-07-29 15:02:55.381142388 +0000 UTC m=+99.289990752 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls") pod "image-registry-746cbbbd85-5hv56" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf") : secret "image-registry-tls" not found Jul 29 15:02:23.425576 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:23.425533 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:23.481915 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:23.481883 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:02:23.482104 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:23.481998 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 15:02:23.482104 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:23.482065 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls podName:bfbfaa07-7732-4510-bfad-041401ba0c43 nodeName:}" failed. No retries permitted until 2026-07-29 15:02:55.482045579 +0000 UTC m=+99.390893938 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls") pod "dns-default-7dqs7" (UID: "bfbfaa07-7732-4510-bfad-041401ba0c43") : secret "dns-default-metrics-tls" not found Jul 29 15:02:23.526683 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:23.526643 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:26.872256 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:26.872212 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qc8sd" event={"ID":"e7b52735-d789-4074-9c97-2e5bdb782caa","Type":"ContainerStarted","Data":"56b28cde162b19b891db0d6a8e859526a24ef9cb1091ce52e95c1343a6b9ceb3"} Jul 29 15:02:26.873560 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:26.873526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2wrlr" event={"ID":"eeb6b594-9641-4f08-875d-6c6d1eb68cb1","Type":"ContainerStarted","Data":"10e0f6f53751dc8a8d84eb92198fb4cede4820934fd7b3fe0c8ffeb8e1915b3f"} Jul 29 15:02:26.873747 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:26.873731 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:02:26.887400 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:26.887350 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qc8sd" podStartSLOduration=65.510478314 podStartE2EDuration="1m9.887335105s" totalImagesPullingTime="4.376856791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:02:26.885979966 +0000 UTC m=+70.794828340" watchObservedRunningTime="2026-07-29 15:02:26.887335105 +0000 UTC m=+70.796183485" Jul 29 15:02:34.404021 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:34.403984 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:34.482864 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:34.482819 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:34.546290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:34.546258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:36.996656 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:36.996613 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:45.019304 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:45.019272 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:45.101272 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:45.101227 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:55.415401 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:55.415343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:02:55.415857 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:55.415500 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 15:02:55.415857 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:55.415522 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-746cbbbd85-5hv56: secret "image-registry-tls" not found Jul 29 15:02:55.415857 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:55.415579 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls podName:716f69de-0819-40aa-90f4-db4954f5bbcf nodeName:}" failed. No retries permitted until 2026-07-29 15:03:59.415564072 +0000 UTC m=+163.324412433 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls") pod "image-registry-746cbbbd85-5hv56" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf") : secret "image-registry-tls" not found Jul 29 15:02:55.516393 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:55.516356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:02:55.516556 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:55.516510 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 15:02:55.516619 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:02:55.516590 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls podName:bfbfaa07-7732-4510-bfad-041401ba0c43 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:59.51657299 +0000 UTC m=+163.425421349 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls") pod "dns-default-7dqs7" (UID: "bfbfaa07-7732-4510-bfad-041401ba0c43") : secret "dns-default-metrics-tls" not found Jul 29 15:02:57.878388 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:57.878355 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2wrlr" Jul 29 15:02:57.894071 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:02:57.894022 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2wrlr" podStartSLOduration=96.458258556 podStartE2EDuration="1m40.894006364s" totalImagesPullingTime="4.435747808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:02:26.899638434 +0000 UTC m=+70.808486808" watchObservedRunningTime="2026-07-29 15:02:57.894006364 +0000 UTC m=+101.802854753" Jul 29 15:03:12.121183 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.121148 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v"] Jul 29 15:03:12.123098 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.123083 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:12.128462 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.128430 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 29 15:03:12.128610 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.128564 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 29 15:03:12.129037 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.128699 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 29 15:03:12.135730 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.135714 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-l8dz9\"" Jul 29 15:03:12.151698 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.151674 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v"] Jul 29 15:03:12.232875 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.232838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:12.233040 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.232906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9npwp\" (UniqueName: \"kubernetes.io/projected/7ea941fc-41f7-4315-bace-1d3345c99228-kube-api-access-9npwp\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:12.333244 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.333201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9npwp\" (UniqueName: \"kubernetes.io/projected/7ea941fc-41f7-4315-bace-1d3345c99228-kube-api-access-9npwp\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:12.333436 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.333268 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:12.333436 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:12.333398 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 29 15:03:12.333549 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:12.333465 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls podName:7ea941fc-41f7-4315-bace-1d3345c99228 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:12.833445286 +0000 UTC m=+116.742293651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls") pod "cluster-samples-operator-6748489656-vwm9v" (UID: "7ea941fc-41f7-4315-bace-1d3345c99228") : secret "samples-operator-tls" not found Jul 29 15:03:12.373694 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.373631 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9npwp\" (UniqueName: \"kubernetes.io/projected/7ea941fc-41f7-4315-bace-1d3345c99228-kube-api-access-9npwp\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:12.672977 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.672896 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:12.802928 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.802893 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:12.836959 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:12.836922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:12.837109 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:12.837063 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 29 15:03:12.837155 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:12.837121 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls podName:7ea941fc-41f7-4315-bace-1d3345c99228 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:13.837107272 +0000 UTC m=+117.745955631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls") pod "cluster-samples-operator-6748489656-vwm9v" (UID: "7ea941fc-41f7-4315-bace-1d3345c99228") : secret "samples-operator-tls" not found Jul 29 15:03:13.844926 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:13.844878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:13.845328 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:13.845034 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 29 15:03:13.845328 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:13.845112 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls podName:7ea941fc-41f7-4315-bace-1d3345c99228 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:15.845095494 +0000 UTC m=+119.753943857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls") pod "cluster-samples-operator-6748489656-vwm9v" (UID: "7ea941fc-41f7-4315-bace-1d3345c99228") : secret "samples-operator-tls" not found Jul 29 15:03:15.112846 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:15.112812 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:15.212962 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:15.212904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:15.859086 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:15.859054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:15.859260 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:15.859162 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 29 15:03:15.859260 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:15.859214 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls podName:7ea941fc-41f7-4315-bace-1d3345c99228 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:19.859200706 +0000 UTC m=+123.768049066 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls") pod "cluster-samples-operator-6748489656-vwm9v" (UID: "7ea941fc-41f7-4315-bace-1d3345c99228") : secret "samples-operator-tls" not found Jul 29 15:03:16.101029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:16.100983 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:16.218786 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:16.218699 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:16.508411 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:16.508370 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:16.601949 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:16.601913 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:16.807740 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:16.807663 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:16.902626 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:16.902586 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:17.095772 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.095695 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w4784"] Jul 29 15:03:17.099401 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.099369 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.102050 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.101896 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 29 15:03:17.102050 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.101898 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 29 15:03:17.102050 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.101957 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ddpzv\"" Jul 29 15:03:17.106424 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.106409 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 29 15:03:17.111549 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.111535 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 29 15:03:17.125977 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.125953 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w4784"] Jul 29 15:03:17.170097 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.170071 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v4n4\" (UniqueName: \"kubernetes.io/projected/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-api-access-6v4n4\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.170217 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.170116 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-crio-socket\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.170217 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.170137 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.170217 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.170162 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-data-volume\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.170217 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.170178 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.178838 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.178810 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:17.271175 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.271139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v4n4\" (UniqueName: \"kubernetes.io/projected/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-api-access-6v4n4\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.271625 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.271200 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-crio-socket\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.271625 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.271224 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.271625 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.271255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-data-volume\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.271625 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.271278 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.271625 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:17.271330 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 15:03:17.271625 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.271382 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-crio-socket\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.271625 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:17.271403 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm podName:d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:17.771383579 +0000 UTC m=+121.680231937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w4784" (UID: "d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514") : configmap "kube-rbac-proxy" not found Jul 29 15:03:17.271625 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:17.271445 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 15:03:17.271625 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:17.271509 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls podName:d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:17.771492609 +0000 UTC m=+121.680340968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w4784" (UID: "d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514") : secret "insights-runtime-extractor-tls" not found Jul 29 15:03:17.271625 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.271619 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-data-volume\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.282222 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.282197 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v4n4\" (UniqueName: \"kubernetes.io/projected/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-api-access-6v4n4\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.362823 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.362758 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:17.775760 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.775729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.776021 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.775764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:17.776021 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:17.775857 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 15:03:17.776021 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:17.775876 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 15:03:17.776021 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:17.775917 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls podName:d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:18.775902465 +0000 UTC m=+122.684750824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w4784" (UID: "d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514") : secret "insights-runtime-extractor-tls" not found Jul 29 15:03:17.776021 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:17.775954 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm podName:d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:18.775933888 +0000 UTC m=+122.684782247 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w4784" (UID: "d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514") : configmap "kube-rbac-proxy" not found Jul 29 15:03:17.784725 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.784701 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv"] Jul 29 15:03:17.786556 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.786542 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv" Jul 29 15:03:17.790243 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.790209 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 29 15:03:17.790411 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.790395 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gf2j4\"" Jul 29 15:03:17.791266 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.791249 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 29 15:03:17.805454 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.805428 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv"] Jul 29 15:03:17.876779 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.876748 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdgxv\" (UniqueName: \"kubernetes.io/projected/c78120ec-62bc-4d4c-a4ab-b607b70ff0ea-kube-api-access-qdgxv\") pod \"migrator-6865c9967-csrpv\" (UID: \"c78120ec-62bc-4d4c-a4ab-b607b70ff0ea\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv" Jul 29 15:03:17.977400 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.977367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdgxv\" (UniqueName: \"kubernetes.io/projected/c78120ec-62bc-4d4c-a4ab-b607b70ff0ea-kube-api-access-qdgxv\") pod \"migrator-6865c9967-csrpv\" (UID: \"c78120ec-62bc-4d4c-a4ab-b607b70ff0ea\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv" Jul 29 15:03:17.991340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:17.991319 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdgxv\" (UniqueName: \"kubernetes.io/projected/c78120ec-62bc-4d4c-a4ab-b607b70ff0ea-kube-api-access-qdgxv\") pod \"migrator-6865c9967-csrpv\" (UID: \"c78120ec-62bc-4d4c-a4ab-b607b70ff0ea\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv" Jul 29 15:03:18.095690 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:18.095609 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv" Jul 29 15:03:18.220807 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:18.220778 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv"] Jul 29 15:03:18.224878 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:03:18.224852 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc78120ec_62bc_4d4c_a4ab_b607b70ff0ea.slice/crio-1a53cd9b334460fd7cf194dca58d5d3fc3ef18bac5fffcab69845074790aeb9f WatchSource:0}: Error finding container 1a53cd9b334460fd7cf194dca58d5d3fc3ef18bac5fffcab69845074790aeb9f: Status 404 returned error can't find the container with id 1a53cd9b334460fd7cf194dca58d5d3fc3ef18bac5fffcab69845074790aeb9f Jul 29 15:03:18.785028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:18.784996 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:18.785512 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:18.785039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:18.785512 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:18.785119 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 15:03:18.785512 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:18.785176 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 15:03:18.785512 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:18.785192 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm podName:d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:20.785172096 +0000 UTC m=+124.694020462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w4784" (UID: "d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514") : configmap "kube-rbac-proxy" not found Jul 29 15:03:18.785512 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:18.785209 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls podName:d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:20.785199893 +0000 UTC m=+124.694048253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w4784" (UID: "d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514") : secret "insights-runtime-extractor-tls" not found Jul 29 15:03:18.992981 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:18.992944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv" event={"ID":"c78120ec-62bc-4d4c-a4ab-b607b70ff0ea","Type":"ContainerStarted","Data":"1a53cd9b334460fd7cf194dca58d5d3fc3ef18bac5fffcab69845074790aeb9f"} Jul 29 15:03:19.130907 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:19.130817 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:19.257818 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:19.257782 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:19.272788 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:19.272764 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pp8jk_c20ebebb-b1b8-4d52-b617-b6faa3d4bf13/dns-node-resolver/0.log" Jul 29 15:03:19.761916 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:19.761879 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:19.895148 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:19.895110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:19.895509 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:19.895257 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 29 15:03:19.895509 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:19.895323 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls podName:7ea941fc-41f7-4315-bace-1d3345c99228 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:27.895305432 +0000 UTC m=+131.804153791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls") pod "cluster-samples-operator-6748489656-vwm9v" (UID: "7ea941fc-41f7-4315-bace-1d3345c99228") : secret "samples-operator-tls" not found Jul 29 15:03:19.996978 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:19.996943 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv" event={"ID":"c78120ec-62bc-4d4c-a4ab-b607b70ff0ea","Type":"ContainerStarted","Data":"09e1498468c18a56fdb2d7bdd3ddc1e8e2db19383a25ae85b573779f241423b8"} Jul 29 15:03:19.997154 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:19.996992 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv" event={"ID":"c78120ec-62bc-4d4c-a4ab-b607b70ff0ea","Type":"ContainerStarted","Data":"04932876bf96039767ec93d5a64d6f18082cae6866ad49ba7d601436feddd5f6"} Jul 29 15:03:20.017242 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:20.017148 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-csrpv" podStartSLOduration=1.946292087 podStartE2EDuration="3.017131242s" totalImagesPullingTime="1.070839155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:20.016461144 +0000 UTC m=+123.925309525" watchObservedRunningTime="2026-07-29 15:03:20.017131242 +0000 UTC m=+123.925979654" Jul 29 15:03:20.267448 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:20.267372 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rthsh_30d268aa-dda6-492b-acd6-b14a838c5948/node-ca/0.log" Jul 29 15:03:20.425788 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:20.425752 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:20.576042 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:20.575957 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:20.669186 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:20.669154 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hv4wr_5318ca26-8284-482e-ab92-a3cb709fb40c/serve-healthcheck-canary/0.log" Jul 29 15:03:20.696733 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:20.696700 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:20.802133 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:20.802095 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:20.802340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:20.802216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:20.802340 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:20.802255 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 15:03:20.802340 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:20.802307 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 15:03:20.802340 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:20.802334 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls podName:d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:24.802313648 +0000 UTC m=+128.711162008 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w4784" (UID: "d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514") : secret "insights-runtime-extractor-tls" not found Jul 29 15:03:20.802512 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:20.802359 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm podName:d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:24.802343271 +0000 UTC m=+128.711191632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w4784" (UID: "d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514") : configmap "kube-rbac-proxy" not found Jul 29 15:03:20.806764 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:20.806728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:20.903517 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:20.903426 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:21.163940 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:21.163844 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:21.914653 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:21.914617 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:22.510557 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:22.510525 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:22.726981 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:22.726946 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:22.899041 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:22.898963 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:23.321280 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:23.321250 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:23.712862 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:23.712766 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:23.908381 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:23.908350 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:24.023657 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:24.023616 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:24.299540 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:24.299465 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:24.686277 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:24.686173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:24.783531 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:24.783490 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:24.832404 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:24.832374 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:24.832558 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:24.832415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:24.832558 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:24.832513 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 15:03:24.832558 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:24.832551 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 15:03:24.832685 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:24.832586 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm podName:d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:32.832567541 +0000 UTC m=+136.741415909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w4784" (UID: "d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514") : configmap "kube-rbac-proxy" not found Jul 29 15:03:24.832685 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:24.832623 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls podName:d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:32.832613351 +0000 UTC m=+136.741461710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w4784" (UID: "d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514") : secret "insights-runtime-extractor-tls" not found Jul 29 15:03:24.911303 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:24.911267 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:25.279180 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:25.279143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:25.438405 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:25.438370 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:03:25.438585 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:25.438514 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 29 15:03:25.438585 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:25.438574 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs podName:6c883406-1472-4688-819f-462bbb78df59 nodeName:}" failed. No retries permitted until 2026-07-29 15:05:27.438560165 +0000 UTC m=+251.347408526 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs") pod "network-metrics-daemon-v4svt" (UID: "6c883406-1472-4688-819f-462bbb78df59") : secret "metrics-daemon-secret" not found Jul 29 15:03:25.913966 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:25.913923 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:26.015789 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:26.015754 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:26.490779 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:26.490739 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:26.703652 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:26.703618 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:27.103186 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:27.103149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:27.327148 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:27.327102 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:27.924425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:27.924388 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:27.957791 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:27.957750 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:27.960577 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:27.960553 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ea941fc-41f7-4315-bace-1d3345c99228-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vwm9v\" (UID: \"7ea941fc-41f7-4315-bace-1d3345c99228\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:28.035469 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:28.035439 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-l8dz9\"" Jul 29 15:03:28.042031 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:28.042010 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" Jul 29 15:03:28.179052 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:28.178962 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v"] Jul 29 15:03:28.599071 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:28.599037 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:28.715536 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:28.715505 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:28.830544 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:28.830509 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:29.020327 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:29.020294 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" event={"ID":"7ea941fc-41f7-4315-bace-1d3345c99228","Type":"ContainerStarted","Data":"988801eae448db4bd160e7d22b56b00cf00d41a69f6426907ea35d8ccc601f8c"} Jul 29 15:03:29.146154 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:29.146106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:29.332010 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:29.331905 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:30.835541 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:30.835506 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:30.950348 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:30.950314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:31.027862 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:31.027825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" event={"ID":"7ea941fc-41f7-4315-bace-1d3345c99228","Type":"ContainerStarted","Data":"3f999209147c48446e39acf79a68716b099bb521449a2b9896fe42470273fe07"} Jul 29 15:03:31.028056 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:31.027874 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" event={"ID":"7ea941fc-41f7-4315-bace-1d3345c99228","Type":"ContainerStarted","Data":"bc7a5fe2b9c8edadcf82bae31ea1e03b3764c7a75a81477e7ee4daa482727f83"} Jul 29 15:03:31.066555 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:31.066500 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vwm9v" podStartSLOduration=16.413847775 podStartE2EDuration="19.06648409s" totalImagesPullingTime="2.652636315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:31.065873331 +0000 UTC m=+134.974721712" watchObservedRunningTime="2026-07-29 15:03:31.06648409 +0000 UTC m=+134.975332470" Jul 29 15:03:31.175030 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:31.174948 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:31.374954 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:31.374915 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:31.541733 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:31.541701 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:32.897812 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:32.897777 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:32.898312 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:32.897900 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:32.898509 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:32.898487 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:32.900218 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:32.900191 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w4784\" (UID: \"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514\") " pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:33.007855 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:33.007804 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w4784" Jul 29 15:03:33.175356 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:33.175270 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w4784"] Jul 29 15:03:33.179758 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:03:33.179733 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd45f0b1a_e6c0_4ee2_a0e7_754c2bf67514.slice/crio-da05bec28e51252346b0c9dd46b4fb2822d35b7f1a9253bfe2a5c4eb46bad6f5 WatchSource:0}: Error finding container da05bec28e51252346b0c9dd46b4fb2822d35b7f1a9253bfe2a5c4eb46bad6f5: Status 404 returned error can't find the container with id da05bec28e51252346b0c9dd46b4fb2822d35b7f1a9253bfe2a5c4eb46bad6f5 Jul 29 15:03:34.039245 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:34.039197 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w4784" event={"ID":"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514","Type":"ContainerStarted","Data":"da05bec28e51252346b0c9dd46b4fb2822d35b7f1a9253bfe2a5c4eb46bad6f5"} Jul 29 15:03:37.394296 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:37.394262 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:39.058835 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:39.058805 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w4784" event={"ID":"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514","Type":"ContainerStarted","Data":"f20b4c379a9e45e67125da546246ee77f7c3622ff9c5c265f9ba33a3721e01c8"} Jul 29 15:03:42.073859 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:42.073822 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w4784" event={"ID":"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514","Type":"ContainerStarted","Data":"0b99b8a985ce67596913dace0fead51713bc41669d938e8a1b822c07ab2ad99d"} Jul 29 15:03:45.862079 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:45.862045 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n"] Jul 29 15:03:45.864118 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:45.864095 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n" Jul 29 15:03:45.866809 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:45.866787 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-r42r8\"" Jul 29 15:03:45.867713 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:45.867694 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 29 15:03:45.883887 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:45.883856 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n"] Jul 29 15:03:45.898118 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:45.898085 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:45.912669 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:45.912640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/99897e39-c00b-433e-b6b7-f879a2f58d83-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-rz69n\" (UID: \"99897e39-c00b-433e-b6b7-f879a2f58d83\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n" Jul 29 15:03:45.989842 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:45.989811 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:46.013913 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:46.013880 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/99897e39-c00b-433e-b6b7-f879a2f58d83-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-rz69n\" (UID: \"99897e39-c00b-433e-b6b7-f879a2f58d83\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n" Jul 29 15:03:46.016587 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:46.016554 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/99897e39-c00b-433e-b6b7-f879a2f58d83-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-rz69n\" (UID: \"99897e39-c00b-433e-b6b7-f879a2f58d83\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n" Jul 29 15:03:46.089329 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:46.089293 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w4784" event={"ID":"d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514","Type":"ContainerStarted","Data":"e036721da8edcb03cc717fb56122c8840a8fa57d989073d563e201f01e733dd8"} Jul 29 15:03:46.113379 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:46.113283 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w4784" podStartSLOduration=16.914229687 podStartE2EDuration="29.113264545s" totalImagesPullingTime="12.199034858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:03:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:46.112794538 +0000 UTC m=+150.021642919" watchObservedRunningTime="2026-07-29 15:03:46.113264545 +0000 UTC m=+150.022112956" Jul 29 15:03:46.174187 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:46.174148 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n" Jul 29 15:03:46.303130 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:46.303098 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n"] Jul 29 15:03:46.308350 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:03:46.308319 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99897e39_c00b_433e_b6b7_f879a2f58d83.slice/crio-87e4cefa1c9c98ff0de7a31673dcf763ddba97deaea1882fedf7aec8ec74e4fb WatchSource:0}: Error finding container 87e4cefa1c9c98ff0de7a31673dcf763ddba97deaea1882fedf7aec8ec74e4fb: Status 404 returned error can't find the container with id 87e4cefa1c9c98ff0de7a31673dcf763ddba97deaea1882fedf7aec8ec74e4fb Jul 29 15:03:47.094149 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:47.094117 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n" event={"ID":"99897e39-c00b-433e-b6b7-f879a2f58d83","Type":"ContainerStarted","Data":"87e4cefa1c9c98ff0de7a31673dcf763ddba97deaea1882fedf7aec8ec74e4fb"} Jul 29 15:03:48.098714 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:48.098675 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n" event={"ID":"99897e39-c00b-433e-b6b7-f879a2f58d83","Type":"ContainerStarted","Data":"5f6f47a012ce6314f7076b2475fa9dbaaa54177e8ac67b5032576f020b091ec8"} Jul 29 15:03:48.099180 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:48.098889 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n" Jul 29 15:03:48.103348 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:48.103321 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n" Jul 29 15:03:48.126179 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:48.126134 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-rz69n" podStartSLOduration=2.029947879 podStartE2EDuration="3.126120904s" totalImagesPullingTime="1.096173025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:03:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:48.125722214 +0000 UTC m=+152.034570598" watchObservedRunningTime="2026-07-29 15:03:48.126120904 +0000 UTC m=+152.034969284" Jul 29 15:03:48.950153 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:48.950115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:49.049919 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:49.049884 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:49.156292 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:49.156264 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:49.901558 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:49.901517 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:49.963048 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:49.963018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:50.843375 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:50.843341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:52.178194 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.178159 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:52.416752 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.416717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:52.507150 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.507119 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wnppj"] Jul 29 15:03:52.509640 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.509624 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.513387 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.513366 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 29 15:03:52.514177 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.514148 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 29 15:03:52.514296 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.514223 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 29 15:03:52.514296 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.514285 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 29 15:03:52.514469 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.514456 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 29 15:03:52.514827 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.514807 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nmr22\"" Jul 29 15:03:52.514827 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.514823 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 29 15:03:52.550917 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.550873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:52.566408 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.566379 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/39569564-987e-4f48-a954-a76f1f1d5512-root\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.566408 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.566409 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-tls\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.566649 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.566430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39569564-987e-4f48-a954-a76f1f1d5512-metrics-client-ca\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.566649 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.566458 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39569564-987e-4f48-a954-a76f1f1d5512-sys\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.566649 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.566479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.566649 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.566583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqrjc\" (UniqueName: \"kubernetes.io/projected/39569564-987e-4f48-a954-a76f1f1d5512-kube-api-access-nqrjc\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.566649 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.566643 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-textfile\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.566862 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.566695 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-accelerators-collector-config\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.566862 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.566726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-wtmp\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.645478 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.645444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:52.667842 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.667815 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-textfile\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.667996 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.667865 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-accelerators-collector-config\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.667996 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.667887 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-wtmp\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668072 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668035 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/39569564-987e-4f48-a954-a76f1f1d5512-root\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668072 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668055 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-wtmp\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668065 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-tls\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668114 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/39569564-987e-4f48-a954-a76f1f1d5512-root\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668142 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39569564-987e-4f48-a954-a76f1f1d5512-metrics-client-ca\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668281 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668152 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39569564-987e-4f48-a954-a76f1f1d5512-sys\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668281 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668281 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668203 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-textfile\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668281 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqrjc\" (UniqueName: \"kubernetes.io/projected/39569564-987e-4f48-a954-a76f1f1d5512-kube-api-access-nqrjc\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668281 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39569564-987e-4f48-a954-a76f1f1d5512-sys\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668281 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:52.668245 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 29 15:03:52.668512 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:52.668429 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-tls podName:39569564-987e-4f48-a954-a76f1f1d5512 nodeName:}" failed. No retries permitted until 2026-07-29 15:03:53.168410644 +0000 UTC m=+157.077259033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-tls") pod "node-exporter-wnppj" (UID: "39569564-987e-4f48-a954-a76f1f1d5512") : secret "node-exporter-tls" not found Jul 29 15:03:52.668614 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668574 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-accelerators-collector-config\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.668705 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.668684 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39569564-987e-4f48-a954-a76f1f1d5512-metrics-client-ca\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.673531 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.673509 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.683047 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.683024 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqrjc\" (UniqueName: \"kubernetes.io/projected/39569564-987e-4f48-a954-a76f1f1d5512-kube-api-access-nqrjc\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:52.889501 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:52.889420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:53.171993 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.171908 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-tls\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:53.174313 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.174282 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39569564-987e-4f48-a954-a76f1f1d5512-node-exporter-tls\") pod \"node-exporter-wnppj\" (UID: \"39569564-987e-4f48-a954-a76f1f1d5512\") " pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:53.418790 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.418754 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wnppj" Jul 29 15:03:53.430585 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:03:53.430546 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39569564_987e_4f48_a954_a76f1f1d5512.slice/crio-e183d3fa8dcb4c538f7ac6aa4eb04c426e2e07376bb8175eafdbebcaf11f8100 WatchSource:0}: Error finding container e183d3fa8dcb4c538f7ac6aa4eb04c426e2e07376bb8175eafdbebcaf11f8100: Status 404 returned error can't find the container with id e183d3fa8dcb4c538f7ac6aa4eb04c426e2e07376bb8175eafdbebcaf11f8100 Jul 29 15:03:53.615364 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.615326 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 15:03:53.618360 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.618344 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.622010 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.621989 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 29 15:03:53.622387 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.622369 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 29 15:03:53.622543 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.622531 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 29 15:03:53.622679 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.622662 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 29 15:03:53.622881 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.622865 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 29 15:03:53.622936 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.622868 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 29 15:03:53.623035 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.623020 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xgzx9\"" Jul 29 15:03:53.623083 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.623050 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 29 15:03:53.623273 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.623257 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 29 15:03:53.627804 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.627785 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 29 15:03:53.639278 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.639253 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 15:03:53.676534 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676502 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.676534 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.676741 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676557 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-config-volume\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.676741 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676574 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-web-config\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.676741 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676615 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52xn4\" (UniqueName: \"kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-kube-api-access-52xn4\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.676741 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676663 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.676741 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.676741 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.677048 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676746 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-config-out\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.677048 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.677048 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676895 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.677048 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676938 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.677048 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.676989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778167 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778141 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778167 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778172 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778426 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778426 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778223 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778426 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778426 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-config-volume\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778426 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-web-config\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778426 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52xn4\" (UniqueName: \"kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-kube-api-access-52xn4\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778426 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778353 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778426 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778426 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778422 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778899 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778439 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-config-out\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.778899 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.779058 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.778911 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.779228 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.779201 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.780002 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.779977 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.781511 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.781411 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.781511 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.781471 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-config-volume\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.781858 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.781834 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.781953 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.781930 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.782217 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.782194 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.782313 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.782247 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-config-out\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.782442 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.782426 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-web-config\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.782827 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.782806 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.783166 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.783146 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.790898 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.790874 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52xn4\" (UniqueName: \"kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-kube-api-access-52xn4\") pod \"alertmanager-main-0\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:53.927360 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:53.927326 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:03:54.089423 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.089369 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 15:03:54.092216 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:03:54.092185 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4796dd84_460a_413f_a332_78134859dcb6.slice/crio-d38e812107d9a8e419829de72a257517ce8e2236d0a8edaf16297e5e74f76d45 WatchSource:0}: Error finding container d38e812107d9a8e419829de72a257517ce8e2236d0a8edaf16297e5e74f76d45: Status 404 returned error can't find the container with id d38e812107d9a8e419829de72a257517ce8e2236d0a8edaf16297e5e74f76d45 Jul 29 15:03:54.120336 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.120299 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerStarted","Data":"d38e812107d9a8e419829de72a257517ce8e2236d0a8edaf16297e5e74f76d45"} Jul 29 15:03:54.121451 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.121426 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wnppj" event={"ID":"39569564-987e-4f48-a954-a76f1f1d5512","Type":"ContainerStarted","Data":"e183d3fa8dcb4c538f7ac6aa4eb04c426e2e07376bb8175eafdbebcaf11f8100"} Jul 29 15:03:54.449699 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.449585 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:54.535832 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:54.535787 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" podUID="716f69de-0819-40aa-90f4-db4954f5bbcf" Jul 29 15:03:54.555256 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.555210 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d677d8db9-rcqvf"] Jul 29 15:03:54.557516 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.557501 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.562637 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.562589 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 29 15:03:54.562772 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.562710 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pg8h2\"" Jul 29 15:03:54.562910 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.562895 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 29 15:03:54.567847 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.567799 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f1rtgsg26g1ci\"" Jul 29 15:03:54.567974 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.567960 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 29 15:03:54.568336 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.568322 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 29 15:03:54.569161 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.569147 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 29 15:03:54.583668 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.583623 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d677d8db9-rcqvf"] Jul 29 15:03:54.648315 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:54.648232 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7dqs7" podUID="bfbfaa07-7732-4510-bfad-041401ba0c43" Jul 29 15:03:54.686227 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.686188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.686404 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.686235 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-grpc-tls\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.686404 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.686306 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.686513 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.686415 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.686513 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.686448 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n27f8\" (UniqueName: \"kubernetes.io/projected/c6288dc9-fc9c-45eb-bded-aece30db7e75-kube-api-access-n27f8\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.686513 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.686472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.686650 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.686505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-tls\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.686650 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.686574 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6288dc9-fc9c-45eb-bded-aece30db7e75-metrics-client-ca\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.787097 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.787045 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.787268 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.787103 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-tls\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.787268 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.787130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6288dc9-fc9c-45eb-bded-aece30db7e75-metrics-client-ca\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.787268 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.787199 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.787430 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.787305 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-grpc-tls\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.787430 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.787348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.787530 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.787437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.787530 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.787465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n27f8\" (UniqueName: \"kubernetes.io/projected/c6288dc9-fc9c-45eb-bded-aece30db7e75-kube-api-access-n27f8\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.788002 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.787974 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6288dc9-fc9c-45eb-bded-aece30db7e75-metrics-client-ca\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.790252 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.790229 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-tls\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.790366 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.790310 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.790803 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.790781 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.791382 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.791362 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.791545 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.791525 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.791915 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.791896 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6288dc9-fc9c-45eb-bded-aece30db7e75-secret-grpc-tls\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.799740 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.799716 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n27f8\" (UniqueName: \"kubernetes.io/projected/c6288dc9-fc9c-45eb-bded-aece30db7e75-kube-api-access-n27f8\") pod \"thanos-querier-5d677d8db9-rcqvf\" (UID: \"c6288dc9-fc9c-45eb-bded-aece30db7e75\") " pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:54.867091 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:54.867053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:55.127742 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:55.127700 2562 generic.go:356] "Generic (PLEG): container finished" podID="39569564-987e-4f48-a954-a76f1f1d5512" containerID="00e59043624a946d988b9b10174173e6f9ed27d8f5fa32a2b4a414043cc3c214" exitCode=0 Jul 29 15:03:55.127890 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:55.127779 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7dqs7" Jul 29 15:03:55.127890 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:55.127808 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wnppj" event={"ID":"39569564-987e-4f48-a954-a76f1f1d5512","Type":"ContainerDied","Data":"00e59043624a946d988b9b10174173e6f9ed27d8f5fa32a2b4a414043cc3c214"} Jul 29 15:03:55.128262 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:55.128068 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:03:55.209634 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:55.209585 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d677d8db9-rcqvf"] Jul 29 15:03:55.249117 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:03:55.249085 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6288dc9_fc9c_45eb_bded_aece30db7e75.slice/crio-1f6a0c3165140c07aef7497b0c7d701e2e6c6b03d9f6ad0867f01b7a92a3e6be WatchSource:0}: Error finding container 1f6a0c3165140c07aef7497b0c7d701e2e6c6b03d9f6ad0867f01b7a92a3e6be: Status 404 returned error can't find the container with id 1f6a0c3165140c07aef7497b0c7d701e2e6c6b03d9f6ad0867f01b7a92a3e6be Jul 29 15:03:55.584110 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:03:55.584066 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-v4svt" podUID="6c883406-1472-4688-819f-462bbb78df59" Jul 29 15:03:56.031627 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.031559 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:56.125997 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.125962 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:56.134033 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.133996 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wnppj" event={"ID":"39569564-987e-4f48-a954-a76f1f1d5512","Type":"ContainerStarted","Data":"a54a84e971aa379aa8a8006471138d262502d4501d003f707d8e1e7c7e01db27"} Jul 29 15:03:56.134191 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.134069 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wnppj" event={"ID":"39569564-987e-4f48-a954-a76f1f1d5512","Type":"ContainerStarted","Data":"7913b955c2add9be572a8898908f9a26e9447b8a70bd9b0fcd90af1b95b838ef"} Jul 29 15:03:56.135817 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.135789 2562 generic.go:356] "Generic (PLEG): container finished" podID="4796dd84-460a-413f-a332-78134859dcb6" containerID="40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458" exitCode=0 Jul 29 15:03:56.135947 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.135875 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerDied","Data":"40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458"} Jul 29 15:03:56.137233 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.137210 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" event={"ID":"c6288dc9-fc9c-45eb-bded-aece30db7e75","Type":"ContainerStarted","Data":"1f6a0c3165140c07aef7497b0c7d701e2e6c6b03d9f6ad0867f01b7a92a3e6be"} Jul 29 15:03:56.175491 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.175427 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wnppj" podStartSLOduration=3.197434265 podStartE2EDuration="4.175406023s" totalImagesPullingTime="903.832324ms" totalInitContainerRuntime="74.139434ms" isStatefulPod=true podCreationTimestamp="2026-07-29 15:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:56.175144324 +0000 UTC m=+160.083992733" watchObservedRunningTime="2026-07-29 15:03:56.175406023 +0000 UTC m=+160.084254406" Jul 29 15:03:56.351990 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.351904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:56.489885 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.489843 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:56.665678 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:56.665530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:57.213982 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:57.213955 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:57.312169 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:57.312121 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:57.424760 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:57.424729 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:57.533329 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:57.533292 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:57.834892 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:57.834812 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:58.147804 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:58.147773 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" event={"ID":"c6288dc9-fc9c-45eb-bded-aece30db7e75","Type":"ContainerStarted","Data":"32c779abf59d64dfc135b904119bcc863ee4a55d64ee74346a2da634f1b130b8"} Jul 29 15:03:58.147907 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:58.147863 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" event={"ID":"c6288dc9-fc9c-45eb-bded-aece30db7e75","Type":"ContainerStarted","Data":"0106caf3ee476e4e1fc9ca81a0447a5105ecc2a10c5736156027e7f755ba2045"} Jul 29 15:03:58.147907 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:58.147892 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" event={"ID":"c6288dc9-fc9c-45eb-bded-aece30db7e75","Type":"ContainerStarted","Data":"457e1e138f90e5af1cfa86b4ce9d9f0f2222bf9a3a260406375ca7e1fa8aaaae"} Jul 29 15:03:58.824931 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:58.824896 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:58.918548 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:58.918520 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:59.024235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.024148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:59.024800 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.024783 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 15:03:59.028295 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.028280 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.036307 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.036287 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 29 15:03:59.041244 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.041229 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ao19e8qcnh73h\"" Jul 29 15:03:59.041471 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.041456 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 29 15:03:59.042235 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.042217 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jln5z\"" Jul 29 15:03:59.042323 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.042310 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 29 15:03:59.042400 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.042387 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 29 15:03:59.042681 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.042667 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 29 15:03:59.042748 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.042707 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 29 15:03:59.042748 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.042738 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 29 15:03:59.042815 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.042755 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 29 15:03:59.044188 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.044173 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 29 15:03:59.044317 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.044200 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 29 15:03:59.044317 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.044254 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 29 15:03:59.044507 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.044484 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 29 15:03:59.046828 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.046811 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 29 15:03:59.047399 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.047374 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 29 15:03:59.048719 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.048702 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 29 15:03:59.108007 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.107931 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 15:03:59.133802 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.133769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.133946 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.133819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.133946 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.133841 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.133946 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.133905 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134041 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.133955 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134041 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.133976 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134041 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.133993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134125 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134125 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134066 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134125 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134206 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134127 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-web-config\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134206 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134156 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134206 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134179 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134206 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134384 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134217 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134384 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-config\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134473 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134388 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-config-out\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134473 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134473 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.134629 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.134487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmflb\" (UniqueName: \"kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-kube-api-access-bmflb\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.154290 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.154251 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerStarted","Data":"127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a"} Jul 29 15:03:59.154430 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.154334 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerStarted","Data":"54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3"} Jul 29 15:03:59.154430 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.154373 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerStarted","Data":"641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50"} Jul 29 15:03:59.154430 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.154410 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerStarted","Data":"d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d"} Jul 29 15:03:59.154616 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.154451 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerStarted","Data":"5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b"} Jul 29 15:03:59.154616 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.154484 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerStarted","Data":"a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3"} Jul 29 15:03:59.157158 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.157137 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" event={"ID":"c6288dc9-fc9c-45eb-bded-aece30db7e75","Type":"ContainerStarted","Data":"9e400deeeaeb6f298f426d2bea70a01707b894b43250bc24759b93d5d4ae8467"} Jul 29 15:03:59.157248 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.157195 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" event={"ID":"c6288dc9-fc9c-45eb-bded-aece30db7e75","Type":"ContainerStarted","Data":"528917f0605dfbe81ca683e5ca6f89578cd9f339091d7ee70c62be9842d67406"} Jul 29 15:03:59.157248 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.157222 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" event={"ID":"c6288dc9-fc9c-45eb-bded-aece30db7e75","Type":"ContainerStarted","Data":"8f728bebd7c522879ec91669610f786dea121656ac84d9210294ca42cb5f3e46"} Jul 29 15:03:59.157307 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.157253 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:03:59.190080 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.190033 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.213894564 podStartE2EDuration="6.190020482s" totalImagesPullingTime="2.950831495s" totalInitContainerRuntime="25.294423ms" isStatefulPod=false podCreationTimestamp="2026-07-29 15:03:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:59.189036315 +0000 UTC m=+163.097884717" watchObservedRunningTime="2026-07-29 15:03:59.190020482 +0000 UTC m=+163.098868865" Jul 29 15:03:59.224229 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.224180 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" podStartSLOduration=2.217497936 podStartE2EDuration="5.224159058s" totalImagesPullingTime="3.006661122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:03:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:59.221712399 +0000 UTC m=+163.130560784" watchObservedRunningTime="2026-07-29 15:03:59.224159058 +0000 UTC m=+163.133007440" Jul 29 15:03:59.235237 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.235337 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235243 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.235337 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235283 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.235458 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235434 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-web-config\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.235518 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.235571 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.235571 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235550 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.235697 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.235765 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-config\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.235827 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-config-out\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.235878 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.235841 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.236212 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236190 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.236379 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmflb\" (UniqueName: \"kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-kube-api-access-bmflb\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.236462 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236413 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.236524 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236466 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.236524 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236501 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.236651 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236549 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.236705 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236651 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.236705 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236656 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.236847 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.236847 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236751 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.237060 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236904 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.237060 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.236979 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.238949 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.237702 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.238949 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.238490 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.239103 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.238977 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.239447 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.239418 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.240179 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.240151 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.241335 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.241309 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.242378 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.242196 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-config\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.242378 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.242336 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.242990 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.242679 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-web-config\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.242990 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.242682 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.243143 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.243000 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-config-out\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.243143 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.243105 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.243143 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.243114 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.243246 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.243136 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.243451 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.243435 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.243845 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.243828 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.258182 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.258162 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmflb\" (UniqueName: \"kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-kube-api-access-bmflb\") pod \"prometheus-k8s-0\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.337409 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.337375 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:03:59.438318 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.438281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:03:59.440645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.440617 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"image-registry-746cbbbd85-5hv56\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:03:59.482356 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.482321 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 15:03:59.486990 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:03:59.486963 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012e5799_a1c2_4f65_9c83_99c036996d11.slice/crio-ac5d001eca8921f6214efdd6a44cb2e9fc22fca75fe020bd8d65d598d3620232 WatchSource:0}: Error finding container ac5d001eca8921f6214efdd6a44cb2e9fc22fca75fe020bd8d65d598d3620232: Status 404 returned error can't find the container with id ac5d001eca8921f6214efdd6a44cb2e9fc22fca75fe020bd8d65d598d3620232 Jul 29 15:03:59.539544 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.539519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:03:59.541787 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.541766 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfbfaa07-7732-4510-bfad-041401ba0c43-metrics-tls\") pod \"dns-default-7dqs7\" (UID: \"bfbfaa07-7732-4510-bfad-041401ba0c43\") " pod="openshift-dns/dns-default-7dqs7" Jul 29 15:03:59.631915 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.631883 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w95m5\"" Jul 29 15:03:59.632071 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.631990 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tlvgb\"" Jul 29 15:03:59.639584 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.639532 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:03:59.639655 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.639634 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7dqs7" Jul 29 15:03:59.782865 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.782832 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7dqs7"] Jul 29 15:03:59.787641 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:03:59.787611 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfbfaa07_7732_4510_bfad_041401ba0c43.slice/crio-1bdb9068714d409370000a180c2aa79e78e32feeea0d75efde71f0b7d3ff6026 WatchSource:0}: Error finding container 1bdb9068714d409370000a180c2aa79e78e32feeea0d75efde71f0b7d3ff6026: Status 404 returned error can't find the container with id 1bdb9068714d409370000a180c2aa79e78e32feeea0d75efde71f0b7d3ff6026 Jul 29 15:03:59.804461 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:03:59.804434 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-746cbbbd85-5hv56"] Jul 29 15:03:59.807139 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:03:59.807115 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod716f69de_0819_40aa_90f4_db4954f5bbcf.slice/crio-be7c97236e14a53c8cf821b6d67d745cff3de4f710e8be141c7c7f4fb788c999 WatchSource:0}: Error finding container be7c97236e14a53c8cf821b6d67d745cff3de4f710e8be141c7c7f4fb788c999: Status 404 returned error can't find the container with id be7c97236e14a53c8cf821b6d67d745cff3de4f710e8be141c7c7f4fb788c999 Jul 29 15:04:00.162172 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:00.162138 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7dqs7" event={"ID":"bfbfaa07-7732-4510-bfad-041401ba0c43","Type":"ContainerStarted","Data":"1bdb9068714d409370000a180c2aa79e78e32feeea0d75efde71f0b7d3ff6026"} Jul 29 15:04:00.163755 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:00.163724 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" event={"ID":"716f69de-0819-40aa-90f4-db4954f5bbcf","Type":"ContainerStarted","Data":"3ec0e9cdd8203ff29bb8105313a51ac2a5d416b9cd66471ed82c4185b77634a0"} Jul 29 15:04:00.163903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:00.163774 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" event={"ID":"716f69de-0819-40aa-90f4-db4954f5bbcf","Type":"ContainerStarted","Data":"be7c97236e14a53c8cf821b6d67d745cff3de4f710e8be141c7c7f4fb788c999"} Jul 29 15:04:00.163903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:00.163818 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:04:00.165425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:00.165399 2562 generic.go:356] "Generic (PLEG): container finished" podID="012e5799-a1c2-4f65-9c83-99c036996d11" containerID="25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f" exitCode=0 Jul 29 15:04:00.165529 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:00.165474 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerDied","Data":"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f"} Jul 29 15:04:00.165570 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:00.165527 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerStarted","Data":"ac5d001eca8921f6214efdd6a44cb2e9fc22fca75fe020bd8d65d598d3620232"} Jul 29 15:04:00.187745 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:00.187680 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" podStartSLOduration=163.187661365 podStartE2EDuration="2m43.187661365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:04:00.186254 +0000 UTC m=+164.095102382" watchObservedRunningTime="2026-07-29 15:04:00.187661365 +0000 UTC m=+164.096509747" Jul 29 15:04:00.189488 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:00.189463 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:00.301242 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:00.301211 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:01.187431 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:01.187366 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:01.281078 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:01.281035 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:04.203239 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:04.203194 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7dqs7" event={"ID":"bfbfaa07-7732-4510-bfad-041401ba0c43","Type":"ContainerStarted","Data":"21ace25b9d8a0d37c7573bbf5c83ed4cb810f80330c668ed7d07d1cdecb7d4ba"} Jul 29 15:04:04.203679 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:04.203267 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7dqs7" event={"ID":"bfbfaa07-7732-4510-bfad-041401ba0c43","Type":"ContainerStarted","Data":"7c56799d136e8c1961acfac23dee834f9773d1dcfe5c8fcb3f6d30c374c095ac"} Jul 29 15:04:04.204512 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:04.204146 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7dqs7" Jul 29 15:04:04.209849 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:04.209534 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerStarted","Data":"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11"} Jul 29 15:04:04.209849 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:04.209632 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerStarted","Data":"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e"} Jul 29 15:04:04.209849 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:04.209673 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerStarted","Data":"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a"} Jul 29 15:04:04.237924 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:04.237870 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7dqs7" podStartSLOduration=129.115656283 podStartE2EDuration="2m13.237845826s" totalImagesPullingTime="4.122189543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:01:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:04:04.237355798 +0000 UTC m=+168.146204182" watchObservedRunningTime="2026-07-29 15:04:04.237845826 +0000 UTC m=+168.146694207" Jul 29 15:04:05.171357 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:05.171323 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d677d8db9-rcqvf" Jul 29 15:04:05.216842 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:05.216801 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerStarted","Data":"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e"} Jul 29 15:04:05.217250 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:05.216878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerStarted","Data":"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9"} Jul 29 15:04:05.217250 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:05.216910 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerStarted","Data":"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568"} Jul 29 15:04:05.281535 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:05.281477 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.515164176 podStartE2EDuration="7.281459104s" totalImagesPullingTime="3.750138147s" totalInitContainerRuntime="16.156781ms" isStatefulPod=false podCreationTimestamp="2026-07-29 15:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:04:05.280613355 +0000 UTC m=+169.189461729" watchObservedRunningTime="2026-07-29 15:04:05.281459104 +0000 UTC m=+169.190307485" Jul 29 15:04:08.563617 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:08.563512 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:04:09.337797 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:09.337759 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:04:09.901004 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:09.900972 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:10.004804 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:10.004766 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:12.818690 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:12.818657 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:12.918901 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:12.918838 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:15.223176 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:15.223150 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7dqs7" Jul 29 15:04:16.177907 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:16.177873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:17.882697 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:17.882662 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:17.980488 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:17.980456 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:18.206135 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:18.206037 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:18.263317 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:18.263275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:19.201140 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:19.201104 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-746cbbbd85-5hv56"] Jul 29 15:04:19.205415 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:19.205383 2562 patch_prober.go:28] interesting pod/image-registry-746cbbbd85-5hv56 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 29 15:04:19.205549 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:19.205429 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" podUID="716f69de-0819-40aa-90f4-db4954f5bbcf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 15:04:19.308618 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:19.308567 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:19.445195 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:19.445162 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:19.547179 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:19.547148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:23.414452 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:23.414403 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4796dd84-460a-413f-a332-78134859dcb6/init-config-reloader/0.log" Jul 29 15:04:23.531526 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:23.531492 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:23.614912 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:23.614884 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4796dd84-460a-413f-a332-78134859dcb6/alertmanager/0.log" Jul 29 15:04:23.622099 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:23.622068 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:23.813911 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:23.813883 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4796dd84-460a-413f-a332-78134859dcb6/config-reloader/0.log" Jul 29 15:04:24.015947 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:24.015916 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4796dd84-460a-413f-a332-78134859dcb6/kube-rbac-proxy-web/0.log" Jul 29 15:04:24.214760 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:24.214677 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4796dd84-460a-413f-a332-78134859dcb6/kube-rbac-proxy/0.log" Jul 29 15:04:24.377646 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:24.377581 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:24.414262 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:24.414210 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4796dd84-460a-413f-a332-78134859dcb6/kube-rbac-proxy-metric/0.log" Jul 29 15:04:24.465611 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:24.465514 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:24.616271 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:24.616244 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4796dd84-460a-413f-a332-78134859dcb6/prom-label-proxy/0.log" Jul 29 15:04:27.213816 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:27.213788 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wnppj_39569564-987e-4f48-a954-a76f1f1d5512/init-textfile/0.log" Jul 29 15:04:27.414368 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:27.414333 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wnppj_39569564-987e-4f48-a954-a76f1f1d5512/node-exporter/0.log" Jul 29 15:04:27.613863 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:27.613840 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wnppj_39569564-987e-4f48-a954-a76f1f1d5512/kube-rbac-proxy/0.log" Jul 29 15:04:28.417660 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:28.417630 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_012e5799-a1c2-4f65-9c83-99c036996d11/init-config-reloader/0.log" Jul 29 15:04:28.616908 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:28.616882 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_012e5799-a1c2-4f65-9c83-99c036996d11/prometheus/0.log" Jul 29 15:04:28.816552 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:28.816529 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_012e5799-a1c2-4f65-9c83-99c036996d11/config-reloader/0.log" Jul 29 15:04:29.014845 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:29.014811 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_012e5799-a1c2-4f65-9c83-99c036996d11/thanos-sidecar/0.log" Jul 29 15:04:29.205861 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:29.205787 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:04:29.221622 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:29.221580 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_012e5799-a1c2-4f65-9c83-99c036996d11/kube-rbac-proxy-web/0.log" Jul 29 15:04:29.415209 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:29.415184 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_012e5799-a1c2-4f65-9c83-99c036996d11/kube-rbac-proxy/0.log" Jul 29 15:04:29.614555 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:29.614528 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_012e5799-a1c2-4f65-9c83-99c036996d11/kube-rbac-proxy-thanos/0.log" Jul 29 15:04:30.213834 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:30.213801 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-rz69n_99897e39-c00b-433e-b6b7-f879a2f58d83/prometheus-operator-admission-webhook/0.log" Jul 29 15:04:30.416223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:30.416196 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/thanos-query/0.log" Jul 29 15:04:30.614140 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:30.614116 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/kube-rbac-proxy-web/0.log" Jul 29 15:04:30.817608 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:30.817578 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/kube-rbac-proxy/0.log" Jul 29 15:04:31.015070 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:31.015041 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/prom-label-proxy/0.log" Jul 29 15:04:31.214802 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:31.214774 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/kube-rbac-proxy-rules/0.log" Jul 29 15:04:31.414201 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:31.414124 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/kube-rbac-proxy-metrics/0.log" Jul 29 15:04:32.829852 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:32.829824 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7dqs7_bfbfaa07-7732-4510-bfad-041401ba0c43/dns/0.log" Jul 29 15:04:33.033178 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:33.033150 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7dqs7_bfbfaa07-7732-4510-bfad-041401ba0c43/kube-rbac-proxy/0.log" Jul 29 15:04:34.336293 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:34.336257 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:34.408289 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:34.408246 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:34.416835 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:34.416811 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pp8jk_c20ebebb-b1b8-4d52-b617-b6faa3d4bf13/dns-node-resolver/0.log" Jul 29 15:04:39.078198 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:39.078161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:40.116223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:40.116192 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:40.215274 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:40.215240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:44.220344 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.220286 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" podUID="716f69de-0819-40aa-90f4-db4954f5bbcf" containerName="registry" containerID="cri-o://3ec0e9cdd8203ff29bb8105313a51ac2a5d416b9cd66471ed82c4185b77634a0" gracePeriod=30 Jul 29 15:04:44.368667 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.368630 2562 generic.go:356] "Generic (PLEG): container finished" podID="716f69de-0819-40aa-90f4-db4954f5bbcf" containerID="3ec0e9cdd8203ff29bb8105313a51ac2a5d416b9cd66471ed82c4185b77634a0" exitCode=0 Jul 29 15:04:44.368827 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.368705 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" event={"ID":"716f69de-0819-40aa-90f4-db4954f5bbcf","Type":"ContainerDied","Data":"3ec0e9cdd8203ff29bb8105313a51ac2a5d416b9cd66471ed82c4185b77634a0"} Jul 29 15:04:44.454845 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.454819 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:04:44.540950 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.540915 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-bound-sa-token\") pod \"716f69de-0819-40aa-90f4-db4954f5bbcf\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " Jul 29 15:04:44.540950 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.540956 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pml9\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-kube-api-access-9pml9\") pod \"716f69de-0819-40aa-90f4-db4954f5bbcf\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " Jul 29 15:04:44.541187 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.540993 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/716f69de-0819-40aa-90f4-db4954f5bbcf-ca-trust-extracted\") pod \"716f69de-0819-40aa-90f4-db4954f5bbcf\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " Jul 29 15:04:44.541187 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.541043 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-installation-pull-secrets\") pod \"716f69de-0819-40aa-90f4-db4954f5bbcf\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " Jul 29 15:04:44.541187 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.541108 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-trusted-ca\") pod \"716f69de-0819-40aa-90f4-db4954f5bbcf\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " Jul 29 15:04:44.541187 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.541132 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-certificates\") pod \"716f69de-0819-40aa-90f4-db4954f5bbcf\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " Jul 29 15:04:44.541187 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.541156 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") pod \"716f69de-0819-40aa-90f4-db4954f5bbcf\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " Jul 29 15:04:44.541187 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.541177 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-image-registry-private-configuration\") pod \"716f69de-0819-40aa-90f4-db4954f5bbcf\" (UID: \"716f69de-0819-40aa-90f4-db4954f5bbcf\") " Jul 29 15:04:44.541624 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.541571 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "716f69de-0819-40aa-90f4-db4954f5bbcf" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:04:44.541624 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.541583 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "716f69de-0819-40aa-90f4-db4954f5bbcf" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:04:44.543756 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.543637 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "716f69de-0819-40aa-90f4-db4954f5bbcf" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:04:44.543756 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.543724 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "716f69de-0819-40aa-90f4-db4954f5bbcf" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:04:44.543988 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.543831 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-kube-api-access-9pml9" (OuterVolumeSpecName: "kube-api-access-9pml9") pod "716f69de-0819-40aa-90f4-db4954f5bbcf" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf"). InnerVolumeSpecName "kube-api-access-9pml9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:04:44.544140 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.544114 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "716f69de-0819-40aa-90f4-db4954f5bbcf" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:04:44.544196 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.544169 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "716f69de-0819-40aa-90f4-db4954f5bbcf" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:04:44.550020 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.549999 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/716f69de-0819-40aa-90f4-db4954f5bbcf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "716f69de-0819-40aa-90f4-db4954f5bbcf" (UID: "716f69de-0819-40aa-90f4-db4954f5bbcf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 15:04:44.642206 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.642172 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-installation-pull-secrets\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:04:44.642206 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.642197 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-trusted-ca\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:04:44.642206 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.642207 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-certificates\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:04:44.642206 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.642215 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-registry-tls\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:04:44.642456 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.642225 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/716f69de-0819-40aa-90f4-db4954f5bbcf-image-registry-private-configuration\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:04:44.642456 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.642235 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-bound-sa-token\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:04:44.642456 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.642244 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pml9\" (UniqueName: \"kubernetes.io/projected/716f69de-0819-40aa-90f4-db4954f5bbcf-kube-api-access-9pml9\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:04:44.642456 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:44.642252 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/716f69de-0819-40aa-90f4-db4954f5bbcf-ca-trust-extracted\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:04:45.373864 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:45.373839 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" Jul 29 15:04:45.374334 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:45.373867 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-746cbbbd85-5hv56" event={"ID":"716f69de-0819-40aa-90f4-db4954f5bbcf","Type":"ContainerDied","Data":"be7c97236e14a53c8cf821b6d67d745cff3de4f710e8be141c7c7f4fb788c999"} Jul 29 15:04:45.374334 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:45.373922 2562 scope.go:117] "RemoveContainer" containerID="3ec0e9cdd8203ff29bb8105313a51ac2a5d416b9cd66471ed82c4185b77634a0" Jul 29 15:04:45.399844 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:45.399815 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-746cbbbd85-5hv56"] Jul 29 15:04:45.407839 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:45.407817 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-746cbbbd85-5hv56"] Jul 29 15:04:45.429894 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:45.429851 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:45.530134 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:45.530097 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:46.185961 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:46.185922 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:46.281216 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:46.281188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:46.568907 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:46.568874 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="716f69de-0819-40aa-90f4-db4954f5bbcf" path="/var/lib/kubelet/pods/716f69de-0819-40aa-90f4-db4954f5bbcf/volumes" Jul 29 15:04:49.523167 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:49.523132 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:49.609539 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:49.609504 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:59.337818 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:59.337777 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:04:59.357643 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:59.357615 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:04:59.444489 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:04:59.444466 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:01.160433 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:01.160402 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:01.225834 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:01.225799 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:12.050723 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:12.050685 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:12.249690 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:12.249649 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:13.115731 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.115698 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 15:05:13.116204 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.116002 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="alertmanager" containerID="cri-o://a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3" gracePeriod=120 Jul 29 15:05:13.116204 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.116072 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy-metric" containerID="cri-o://54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3" gracePeriod=120 Jul 29 15:05:13.116204 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.116156 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy-web" containerID="cri-o://d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d" gracePeriod=120 Jul 29 15:05:13.116204 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.116130 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="prom-label-proxy" containerID="cri-o://127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a" gracePeriod=120 Jul 29 15:05:13.116204 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.116187 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="config-reloader" containerID="cri-o://5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b" gracePeriod=120 Jul 29 15:05:13.116458 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.116225 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy" containerID="cri-o://641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50" gracePeriod=120 Jul 29 15:05:13.484291 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.484200 2562 generic.go:356] "Generic (PLEG): container finished" podID="4796dd84-460a-413f-a332-78134859dcb6" containerID="127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a" exitCode=0 Jul 29 15:05:13.484291 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.484228 2562 generic.go:356] "Generic (PLEG): container finished" podID="4796dd84-460a-413f-a332-78134859dcb6" containerID="641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50" exitCode=0 Jul 29 15:05:13.484291 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.484237 2562 generic.go:356] "Generic (PLEG): container finished" podID="4796dd84-460a-413f-a332-78134859dcb6" containerID="5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b" exitCode=0 Jul 29 15:05:13.484291 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.484243 2562 generic.go:356] "Generic (PLEG): container finished" podID="4796dd84-460a-413f-a332-78134859dcb6" containerID="a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3" exitCode=0 Jul 29 15:05:13.484291 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.484265 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerDied","Data":"127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a"} Jul 29 15:05:13.484541 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.484329 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerDied","Data":"641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50"} Jul 29 15:05:13.484541 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.484357 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerDied","Data":"5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b"} Jul 29 15:05:13.484541 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:13.484383 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerDied","Data":"a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3"} Jul 29 15:05:14.362253 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.362229 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.391060 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391014 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-trusted-ca-bundle\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391060 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391052 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-tls-assets\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391097 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52xn4\" (UniqueName: \"kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-kube-api-access-52xn4\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391127 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-metrics-client-ca\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391162 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391218 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-cluster-tls-config\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391249 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-web-config\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391276 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-config-volume\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391340 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391314 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-config-out\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391731 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391350 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-main-tls\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391731 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391379 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-main-db\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391731 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391407 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-web\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391731 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391434 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy\") pod \"4796dd84-460a-413f-a332-78134859dcb6\" (UID: \"4796dd84-460a-413f-a332-78134859dcb6\") " Jul 29 15:05:14.391731 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.391659 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:14.392148 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.392101 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:14.393427 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.393353 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 15:05:14.393427 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.393367 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.393427 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.393405 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4796dd84-460a-413f-a332-78134859dcb6-metrics-client-ca\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.397731 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.397295 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:14.397731 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.397399 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-config-out" (OuterVolumeSpecName: "config-out") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 15:05:14.397998 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.397960 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:14.398491 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.398366 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:14.398491 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.398457 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:05:14.398801 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.398766 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:14.399671 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.399624 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-kube-api-access-52xn4" (OuterVolumeSpecName: "kube-api-access-52xn4") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "kube-api-access-52xn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:05:14.402385 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.402359 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-config-volume" (OuterVolumeSpecName: "config-volume") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:14.402953 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.402926 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:14.409165 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.409136 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-web-config" (OuterVolumeSpecName: "web-config") pod "4796dd84-460a-413f-a332-78134859dcb6" (UID: "4796dd84-460a-413f-a332-78134859dcb6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:14.491930 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.491901 2562 generic.go:356] "Generic (PLEG): container finished" podID="4796dd84-460a-413f-a332-78134859dcb6" containerID="54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3" exitCode=0 Jul 29 15:05:14.491930 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.491924 2562 generic.go:356] "Generic (PLEG): container finished" podID="4796dd84-460a-413f-a332-78134859dcb6" containerID="d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d" exitCode=0 Jul 29 15:05:14.492183 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.491953 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerDied","Data":"54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3"} Jul 29 15:05:14.492183 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.491990 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.492183 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.492030 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerDied","Data":"d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d"} Jul 29 15:05:14.492183 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.492062 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4796dd84-460a-413f-a332-78134859dcb6","Type":"ContainerDied","Data":"d38e812107d9a8e419829de72a257517ce8e2236d0a8edaf16297e5e74f76d45"} Jul 29 15:05:14.492183 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.492092 2562 scope.go:117] "RemoveContainer" containerID="127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a" Jul 29 15:05:14.493829 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493807 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52xn4\" (UniqueName: \"kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-kube-api-access-52xn4\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.493829 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493828 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.494029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493838 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-cluster-tls-config\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.494029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493848 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-web-config\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.494029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493857 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-config-volume\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.494029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493866 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-config-out\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.494029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493873 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-main-tls\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.494029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493884 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4796dd84-460a-413f-a332-78134859dcb6-alertmanager-main-db\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.494029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493892 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.494029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493902 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4796dd84-460a-413f-a332-78134859dcb6-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.494029 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.493910 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4796dd84-460a-413f-a332-78134859dcb6-tls-assets\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:14.500569 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.500554 2562 scope.go:117] "RemoveContainer" containerID="54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3" Jul 29 15:05:14.508408 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.508388 2562 scope.go:117] "RemoveContainer" containerID="641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50" Jul 29 15:05:14.515166 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.515139 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 15:05:14.516191 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.516174 2562 scope.go:117] "RemoveContainer" containerID="d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d" Jul 29 15:05:14.519661 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.519627 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 15:05:14.524032 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.524012 2562 scope.go:117] "RemoveContainer" containerID="5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b" Jul 29 15:05:14.531405 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.531386 2562 scope.go:117] "RemoveContainer" containerID="a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3" Jul 29 15:05:14.540782 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.540764 2562 scope.go:117] "RemoveContainer" containerID="40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458" Jul 29 15:05:14.546838 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.546816 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 15:05:14.547093 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547081 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="716f69de-0819-40aa-90f4-db4954f5bbcf" containerName="registry" Jul 29 15:05:14.547093 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547093 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="716f69de-0819-40aa-90f4-db4954f5bbcf" containerName="registry" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547103 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy-metric" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547109 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy-metric" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547117 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="alertmanager" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547122 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="alertmanager" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547128 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547134 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547141 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="init-config-reloader" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547146 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="init-config-reloader" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547153 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy-web" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547158 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy-web" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547169 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="prom-label-proxy" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547173 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="prom-label-proxy" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547180 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="config-reloader" Jul 29 15:05:14.547205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547184 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="config-reloader" Jul 29 15:05:14.547578 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547222 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="alertmanager" Jul 29 15:05:14.547578 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547229 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="config-reloader" Jul 29 15:05:14.547578 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547234 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy-metric" Jul 29 15:05:14.547578 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547239 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy" Jul 29 15:05:14.547578 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547244 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="716f69de-0819-40aa-90f4-db4954f5bbcf" containerName="registry" Jul 29 15:05:14.547578 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547250 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="kube-rbac-proxy-web" Jul 29 15:05:14.547578 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.547255 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4796dd84-460a-413f-a332-78134859dcb6" containerName="prom-label-proxy" Jul 29 15:05:14.548718 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.548702 2562 scope.go:117] "RemoveContainer" containerID="127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a" Jul 29 15:05:14.548985 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:14.548962 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a\": container with ID starting with 127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a not found: ID does not exist" containerID="127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a" Jul 29 15:05:14.549085 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.548991 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a"} err="failed to get container status \"127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a\": rpc error: code = NotFound desc = could not find container \"127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a\": container with ID starting with 127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a not found: ID does not exist" Jul 29 15:05:14.549085 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.549022 2562 scope.go:117] "RemoveContainer" containerID="54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3" Jul 29 15:05:14.549293 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:14.549274 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3\": container with ID starting with 54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3 not found: ID does not exist" containerID="54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3" Jul 29 15:05:14.549338 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.549300 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3"} err="failed to get container status \"54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3\": rpc error: code = NotFound desc = could not find container \"54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3\": container with ID starting with 54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3 not found: ID does not exist" Jul 29 15:05:14.549338 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.549318 2562 scope.go:117] "RemoveContainer" containerID="641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50" Jul 29 15:05:14.549560 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:14.549544 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50\": container with ID starting with 641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50 not found: ID does not exist" containerID="641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50" Jul 29 15:05:14.549617 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.549566 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50"} err="failed to get container status \"641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50\": rpc error: code = NotFound desc = could not find container \"641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50\": container with ID starting with 641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50 not found: ID does not exist" Jul 29 15:05:14.549617 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.549581 2562 scope.go:117] "RemoveContainer" containerID="d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d" Jul 29 15:05:14.549793 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:14.549775 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d\": container with ID starting with d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d not found: ID does not exist" containerID="d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d" Jul 29 15:05:14.549833 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.549801 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d"} err="failed to get container status \"d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d\": rpc error: code = NotFound desc = could not find container \"d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d\": container with ID starting with d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d not found: ID does not exist" Jul 29 15:05:14.549833 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.549814 2562 scope.go:117] "RemoveContainer" containerID="5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b" Jul 29 15:05:14.550032 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:14.550015 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b\": container with ID starting with 5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b not found: ID does not exist" containerID="5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b" Jul 29 15:05:14.550069 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.550037 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b"} err="failed to get container status \"5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b\": rpc error: code = NotFound desc = could not find container \"5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b\": container with ID starting with 5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b not found: ID does not exist" Jul 29 15:05:14.550069 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.550052 2562 scope.go:117] "RemoveContainer" containerID="a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3" Jul 29 15:05:14.550302 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:14.550283 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3\": container with ID starting with a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3 not found: ID does not exist" containerID="a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3" Jul 29 15:05:14.550349 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.550309 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3"} err="failed to get container status \"a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3\": rpc error: code = NotFound desc = could not find container \"a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3\": container with ID starting with a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3 not found: ID does not exist" Jul 29 15:05:14.550349 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.550322 2562 scope.go:117] "RemoveContainer" containerID="40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458" Jul 29 15:05:14.550554 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:14.550537 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458\": container with ID starting with 40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458 not found: ID does not exist" containerID="40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458" Jul 29 15:05:14.550615 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.550558 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458"} err="failed to get container status \"40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458\": rpc error: code = NotFound desc = could not find container \"40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458\": container with ID starting with 40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458 not found: ID does not exist" Jul 29 15:05:14.550615 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.550573 2562 scope.go:117] "RemoveContainer" containerID="127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a" Jul 29 15:05:14.550819 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.550801 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a"} err="failed to get container status \"127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a\": rpc error: code = NotFound desc = could not find container \"127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a\": container with ID starting with 127d0f4d4b37032459c55c1a7651876d34348554b89d94f76b5207920e94125a not found: ID does not exist" Jul 29 15:05:14.550819 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.550818 2562 scope.go:117] "RemoveContainer" containerID="54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3" Jul 29 15:05:14.550960 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.550944 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.551030 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.551015 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3"} err="failed to get container status \"54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3\": rpc error: code = NotFound desc = could not find container \"54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3\": container with ID starting with 54ac163d5723ef4f85e74b0317088b77c65e0d89cc0fc3b0f1d0947007c5ece3 not found: ID does not exist" Jul 29 15:05:14.551092 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.551031 2562 scope.go:117] "RemoveContainer" containerID="641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50" Jul 29 15:05:14.551453 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.551431 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50"} err="failed to get container status \"641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50\": rpc error: code = NotFound desc = could not find container \"641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50\": container with ID starting with 641e5070993c0db1c6e889bdc034bdc40848596eebf34a10733fe02b16852e50 not found: ID does not exist" Jul 29 15:05:14.551453 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.551453 2562 scope.go:117] "RemoveContainer" containerID="d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d" Jul 29 15:05:14.551751 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.551732 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d"} err="failed to get container status \"d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d\": rpc error: code = NotFound desc = could not find container \"d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d\": container with ID starting with d417a118c70fce9028150a99b646597a8ecd50f76830b2ed1ee3125dd0d90b4d not found: ID does not exist" Jul 29 15:05:14.551751 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.551751 2562 scope.go:117] "RemoveContainer" containerID="5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b" Jul 29 15:05:14.551999 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.551981 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b"} err="failed to get container status \"5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b\": rpc error: code = NotFound desc = could not find container \"5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b\": container with ID starting with 5ff8bca2fd71859111eab5abd7c8ba18422305ed0f098128e990f1202aee993b not found: ID does not exist" Jul 29 15:05:14.552056 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.552001 2562 scope.go:117] "RemoveContainer" containerID="a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3" Jul 29 15:05:14.552223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.552203 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3"} err="failed to get container status \"a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3\": rpc error: code = NotFound desc = could not find container \"a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3\": container with ID starting with a6c560d0dcc20d6898c7d05f52a51ad7998a7e6cf79281eda89d0da2ce8e45d3 not found: ID does not exist" Jul 29 15:05:14.552223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.552223 2562 scope.go:117] "RemoveContainer" containerID="40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458" Jul 29 15:05:14.552408 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.552394 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458"} err="failed to get container status \"40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458\": rpc error: code = NotFound desc = could not find container \"40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458\": container with ID starting with 40a0a4090c3d7b9c6058cd6dd986ce72d13b1f572fc7437f9b020296df7ed458 not found: ID does not exist" Jul 29 15:05:14.553138 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.553122 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 29 15:05:14.553138 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.553134 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 29 15:05:14.553237 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.553126 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 29 15:05:14.553414 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.553397 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 29 15:05:14.553414 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.553425 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 29 15:05:14.553569 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.553431 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 29 15:05:14.553569 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.553429 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 29 15:05:14.553569 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.553506 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xgzx9\"" Jul 29 15:05:14.553569 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.553436 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 29 15:05:14.562186 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.562157 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 29 15:05:14.568069 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.568040 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4796dd84-460a-413f-a332-78134859dcb6" path="/var/lib/kubelet/pods/4796dd84-460a-413f-a332-78134859dcb6/volumes" Jul 29 15:05:14.570205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.570183 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 15:05:14.594997 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.594963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a3a34b7-100b-47ae-86f2-95ae47689c26-config-out\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595141 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595015 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5z9k\" (UniqueName: \"kubernetes.io/projected/7a3a34b7-100b-47ae-86f2-95ae47689c26-kube-api-access-k5z9k\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595141 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595036 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7a3a34b7-100b-47ae-86f2-95ae47689c26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595141 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595085 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a3a34b7-100b-47ae-86f2-95ae47689c26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595236 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595140 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-config-volume\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595236 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595158 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-web-config\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595236 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595177 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595328 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595328 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595270 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595328 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595296 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595328 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595312 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a3a34b7-100b-47ae-86f2-95ae47689c26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595444 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595334 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a3a34b7-100b-47ae-86f2-95ae47689c26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.595444 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.595377 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696122 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696122 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a3a34b7-100b-47ae-86f2-95ae47689c26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696335 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696150 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a3a34b7-100b-47ae-86f2-95ae47689c26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696335 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696171 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696335 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a3a34b7-100b-47ae-86f2-95ae47689c26-config-out\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696335 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5z9k\" (UniqueName: \"kubernetes.io/projected/7a3a34b7-100b-47ae-86f2-95ae47689c26-kube-api-access-k5z9k\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696335 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7a3a34b7-100b-47ae-86f2-95ae47689c26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696335 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a3a34b7-100b-47ae-86f2-95ae47689c26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696335 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696337 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-config-volume\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696689 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-web-config\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696689 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696388 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696689 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.696689 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.696449 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.697181 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.697145 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a3a34b7-100b-47ae-86f2-95ae47689c26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.697284 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.697180 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7a3a34b7-100b-47ae-86f2-95ae47689c26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.697647 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.697585 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a3a34b7-100b-47ae-86f2-95ae47689c26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.699443 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.699414 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a3a34b7-100b-47ae-86f2-95ae47689c26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.699549 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.699431 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.699766 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.699694 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.699766 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.699696 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.699961 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.699939 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-config-volume\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.700028 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.700012 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a3a34b7-100b-47ae-86f2-95ae47689c26-config-out\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.700071 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.700063 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-web-config\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.700498 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.700479 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.701608 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.701578 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a3a34b7-100b-47ae-86f2-95ae47689c26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.704021 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.703995 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5z9k\" (UniqueName: \"kubernetes.io/projected/7a3a34b7-100b-47ae-86f2-95ae47689c26-kube-api-access-k5z9k\") pod \"alertmanager-main-0\" (UID: \"7a3a34b7-100b-47ae-86f2-95ae47689c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:14.862624 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:14.862509 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 15:05:15.011567 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:15.011538 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 15:05:15.015779 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:05:15.015749 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3a34b7_100b_47ae_86f2_95ae47689c26.slice/crio-536866dd9df12f974d49fd7da863120926c65cdf7d9bf5cad3d5d11e214ee588 WatchSource:0}: Error finding container 536866dd9df12f974d49fd7da863120926c65cdf7d9bf5cad3d5d11e214ee588: Status 404 returned error can't find the container with id 536866dd9df12f974d49fd7da863120926c65cdf7d9bf5cad3d5d11e214ee588 Jul 29 15:05:15.497367 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:15.497335 2562 generic.go:356] "Generic (PLEG): container finished" podID="7a3a34b7-100b-47ae-86f2-95ae47689c26" containerID="b602cc7503ac96cdd3b966faa4b9e35fdb9265b1d7b1bb9a7223388cc15a02d4" exitCode=0 Jul 29 15:05:15.497935 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:15.497421 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7a3a34b7-100b-47ae-86f2-95ae47689c26","Type":"ContainerDied","Data":"b602cc7503ac96cdd3b966faa4b9e35fdb9265b1d7b1bb9a7223388cc15a02d4"} Jul 29 15:05:15.497935 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:15.497492 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7a3a34b7-100b-47ae-86f2-95ae47689c26","Type":"ContainerStarted","Data":"536866dd9df12f974d49fd7da863120926c65cdf7d9bf5cad3d5d11e214ee588"} Jul 29 15:05:16.505615 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:16.505563 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7a3a34b7-100b-47ae-86f2-95ae47689c26","Type":"ContainerStarted","Data":"4dfd75b84d6bbfae599c7821bbfa4b478626bbdc86826e95d833c8b9b5cdf8fa"} Jul 29 15:05:16.506116 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:16.505651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7a3a34b7-100b-47ae-86f2-95ae47689c26","Type":"ContainerStarted","Data":"bf85ee5351c67cc25f322c400cb87960644bbb8c5bf5974d461d17da75471ffe"} Jul 29 15:05:16.506116 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:16.505695 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7a3a34b7-100b-47ae-86f2-95ae47689c26","Type":"ContainerStarted","Data":"f96521192c8ffdced59a13b7208559b56a5b80e0385d62c86bf6548eb2bdfd81"} Jul 29 15:05:16.506116 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:16.505724 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7a3a34b7-100b-47ae-86f2-95ae47689c26","Type":"ContainerStarted","Data":"b7dff0079f99880ce53cd31216b2e8cd6c396ae3cf343f1ebe243d837800c04f"} Jul 29 15:05:16.506116 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:16.505748 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7a3a34b7-100b-47ae-86f2-95ae47689c26","Type":"ContainerStarted","Data":"c3690c12495c02f9f000bcf6899e7e46378e8d2dbea2297be368ac6a0e60b074"} Jul 29 15:05:16.506116 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:16.505789 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7a3a34b7-100b-47ae-86f2-95ae47689c26","Type":"ContainerStarted","Data":"7cdbc3b7a5ecb8dd61adc1b14ba19d2f24aa928a4c0aba1bc15617062d712055"} Jul 29 15:05:16.532785 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:16.532733 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.517438922 podStartE2EDuration="2.532716218s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.277296ms" isStatefulPod=false podCreationTimestamp="2026-07-29 15:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:05:16.53152244 +0000 UTC m=+240.440370821" watchObservedRunningTime="2026-07-29 15:05:16.532716218 +0000 UTC m=+240.441564599" Jul 29 15:05:17.142374 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.142334 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f4c7db59c-xflr6"] Jul 29 15:05:17.146777 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.146756 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.150083 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.150058 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 29 15:05:17.150213 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.150060 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 29 15:05:17.150213 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.150101 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 29 15:05:17.150213 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.150116 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 29 15:05:17.150440 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.150425 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 29 15:05:17.150530 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.150515 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kr2m4\"" Jul 29 15:05:17.161615 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.161574 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 29 15:05:17.170897 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.170863 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f4c7db59c-xflr6"] Jul 29 15:05:17.217665 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.217632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-telemeter-client-tls\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.217847 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.217670 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-federate-client-tls\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.217847 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.217711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/619ff693-e355-4bcc-91e9-90b26ef188c6-serving-certs-ca-bundle\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.217847 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.217731 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.217847 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.217815 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/619ff693-e355-4bcc-91e9-90b26ef188c6-metrics-client-ca\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.218032 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.217879 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-secret-telemeter-client\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.218032 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.217935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/619ff693-e355-4bcc-91e9-90b26ef188c6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.218032 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.218010 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88m46\" (UniqueName: \"kubernetes.io/projected/619ff693-e355-4bcc-91e9-90b26ef188c6-kube-api-access-88m46\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.318492 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.318457 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/619ff693-e355-4bcc-91e9-90b26ef188c6-serving-certs-ca-bundle\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.318492 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.318503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.318782 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.318530 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/619ff693-e355-4bcc-91e9-90b26ef188c6-metrics-client-ca\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.318782 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.318572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-secret-telemeter-client\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.318782 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.318643 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/619ff693-e355-4bcc-91e9-90b26ef188c6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.318782 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.318709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88m46\" (UniqueName: \"kubernetes.io/projected/619ff693-e355-4bcc-91e9-90b26ef188c6-kube-api-access-88m46\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.318782 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.318763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-telemeter-client-tls\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.319025 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.318784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-federate-client-tls\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.319386 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.319355 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/619ff693-e355-4bcc-91e9-90b26ef188c6-metrics-client-ca\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.319386 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.319372 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/619ff693-e355-4bcc-91e9-90b26ef188c6-serving-certs-ca-bundle\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.319562 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.319458 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/619ff693-e355-4bcc-91e9-90b26ef188c6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.321437 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.321402 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-secret-telemeter-client\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.321562 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.321545 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-telemeter-client-tls\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.321644 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.321568 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.321745 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.321726 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/619ff693-e355-4bcc-91e9-90b26ef188c6-federate-client-tls\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.327257 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.327235 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88m46\" (UniqueName: \"kubernetes.io/projected/619ff693-e355-4bcc-91e9-90b26ef188c6-kube-api-access-88m46\") pod \"telemeter-client-7f4c7db59c-xflr6\" (UID: \"619ff693-e355-4bcc-91e9-90b26ef188c6\") " pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.436786 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.436690 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 15:05:17.437055 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.437016 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="prometheus" containerID="cri-o://7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a" gracePeriod=600 Jul 29 15:05:17.437055 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.437041 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy" containerID="cri-o://d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9" gracePeriod=600 Jul 29 15:05:17.437223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.437060 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="thanos-sidecar" containerID="cri-o://922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11" gracePeriod=600 Jul 29 15:05:17.437223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.437092 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="config-reloader" containerID="cri-o://3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e" gracePeriod=600 Jul 29 15:05:17.437223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.437106 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy-thanos" containerID="cri-o://21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e" gracePeriod=600 Jul 29 15:05:17.437223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.437196 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy-web" containerID="cri-o://4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568" gracePeriod=600 Jul 29 15:05:17.456224 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.456200 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" Jul 29 15:05:17.599784 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.599747 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f4c7db59c-xflr6"] Jul 29 15:05:17.605951 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:05:17.605909 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod619ff693_e355_4bcc_91e9_90b26ef188c6.slice/crio-c417a82e0c86c5436f7dfd8f794805ea7154b14e186a3d067999034e2e8d014f WatchSource:0}: Error finding container c417a82e0c86c5436f7dfd8f794805ea7154b14e186a3d067999034e2e8d014f: Status 404 returned error can't find the container with id c417a82e0c86c5436f7dfd8f794805ea7154b14e186a3d067999034e2e8d014f Jul 29 15:05:17.684615 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.684578 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:17.722502 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722411 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-thanos-prometheus-http-client-file\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.722502 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722460 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmflb\" (UniqueName: \"kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-kube-api-access-bmflb\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.722502 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722501 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-web-config\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.722793 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722657 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-tls-assets\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.722793 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722699 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-serving-certs-ca-bundle\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.722793 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722727 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.722793 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722754 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-config-out\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.722793 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722786 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-0\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722814 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-1\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722846 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-db\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722887 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-metrics-client-ca\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722919 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-grpc-tls\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722950 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.722978 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-kubelet-serving-ca-bundle\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.723016 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-kube-rbac-proxy\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723342 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.723047 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-2\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723342 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.723075 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-tls\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723342 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.723110 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-trusted-ca-bundle\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723342 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.723178 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-metrics-client-certs\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.723342 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.723206 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-config\") pod \"012e5799-a1c2-4f65-9c83-99c036996d11\" (UID: \"012e5799-a1c2-4f65-9c83-99c036996d11\") " Jul 29 15:05:17.724630 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.724259 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:17.725404 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.724772 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:17.725404 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.725139 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:17.725656 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.725629 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:17.727425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.727140 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-config-out" (OuterVolumeSpecName: "config-out") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 15:05:17.727425 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.727153 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 15:05:17.727763 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.727736 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:17.727853 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.727819 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:05:17.728400 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.728373 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:17.728730 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.728704 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:17.728826 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.728801 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:17.729027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.728929 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:17.729027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.728952 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-kube-api-access-bmflb" (OuterVolumeSpecName: "kube-api-access-bmflb") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "kube-api-access-bmflb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:05:17.729027 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.728993 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:17.729205 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.729172 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-config" (OuterVolumeSpecName: "config") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:17.730158 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.730121 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:17.730815 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.730790 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:17.730906 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.730888 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:17.731309 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.731292 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:17.738101 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.738078 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-web-config" (OuterVolumeSpecName: "web-config") pod "012e5799-a1c2-4f65-9c83-99c036996d11" (UID: "012e5799-a1c2-4f65-9c83-99c036996d11"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 15:05:17.824799 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824760 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.824799 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824791 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-kube-rbac-proxy\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.824799 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824804 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824818 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824831 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824842 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-metrics-client-certs\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824853 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-config\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824866 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824877 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmflb\" (UniqueName: \"kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-kube-api-access-bmflb\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824887 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-web-config\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824900 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/012e5799-a1c2-4f65-9c83-99c036996d11-tls-assets\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824910 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824922 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824934 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-config-out\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824946 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824957 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824968 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/012e5799-a1c2-4f65-9c83-99c036996d11-prometheus-k8s-db\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824979 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/012e5799-a1c2-4f65-9c83-99c036996d11-configmap-metrics-client-ca\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.824990 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-grpc-tls\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:17.825074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:17.825002 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/012e5799-a1c2-4f65-9c83-99c036996d11-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:05:18.518548 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518512 2562 generic.go:356] "Generic (PLEG): container finished" podID="012e5799-a1c2-4f65-9c83-99c036996d11" containerID="21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e" exitCode=0 Jul 29 15:05:18.518548 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518539 2562 generic.go:356] "Generic (PLEG): container finished" podID="012e5799-a1c2-4f65-9c83-99c036996d11" containerID="d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9" exitCode=0 Jul 29 15:05:18.518548 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518546 2562 generic.go:356] "Generic (PLEG): container finished" podID="012e5799-a1c2-4f65-9c83-99c036996d11" containerID="4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568" exitCode=0 Jul 29 15:05:18.518548 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518551 2562 generic.go:356] "Generic (PLEG): container finished" podID="012e5799-a1c2-4f65-9c83-99c036996d11" containerID="922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11" exitCode=0 Jul 29 15:05:18.518548 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518556 2562 generic.go:356] "Generic (PLEG): container finished" podID="012e5799-a1c2-4f65-9c83-99c036996d11" containerID="3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e" exitCode=0 Jul 29 15:05:18.518548 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518561 2562 generic.go:356] "Generic (PLEG): container finished" podID="012e5799-a1c2-4f65-9c83-99c036996d11" containerID="7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a" exitCode=0 Jul 29 15:05:18.518986 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518624 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.518986 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerDied","Data":"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e"} Jul 29 15:05:18.518986 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518705 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerDied","Data":"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9"} Jul 29 15:05:18.518986 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518748 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerDied","Data":"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568"} Jul 29 15:05:18.518986 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518791 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerDied","Data":"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11"} Jul 29 15:05:18.518986 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518802 2562 scope.go:117] "RemoveContainer" containerID="21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e" Jul 29 15:05:18.518986 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518832 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerDied","Data":"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e"} Jul 29 15:05:18.518986 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518874 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerDied","Data":"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a"} Jul 29 15:05:18.518986 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.518917 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"012e5799-a1c2-4f65-9c83-99c036996d11","Type":"ContainerDied","Data":"ac5d001eca8921f6214efdd6a44cb2e9fc22fca75fe020bd8d65d598d3620232"} Jul 29 15:05:18.520579 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.520554 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" event={"ID":"619ff693-e355-4bcc-91e9-90b26ef188c6","Type":"ContainerStarted","Data":"c417a82e0c86c5436f7dfd8f794805ea7154b14e186a3d067999034e2e8d014f"} Jul 29 15:05:18.532376 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.532351 2562 scope.go:117] "RemoveContainer" containerID="d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9" Jul 29 15:05:18.539968 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.539951 2562 scope.go:117] "RemoveContainer" containerID="4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568" Jul 29 15:05:18.547107 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.547081 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 15:05:18.547683 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.547670 2562 scope.go:117] "RemoveContainer" containerID="922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11" Jul 29 15:05:18.550259 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.550240 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 15:05:18.554827 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.554811 2562 scope.go:117] "RemoveContainer" containerID="3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e" Jul 29 15:05:18.561632 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.561615 2562 scope.go:117] "RemoveContainer" containerID="7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a" Jul 29 15:05:18.569513 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.569484 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" path="/var/lib/kubelet/pods/012e5799-a1c2-4f65-9c83-99c036996d11/volumes" Jul 29 15:05:18.570456 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.570442 2562 scope.go:117] "RemoveContainer" containerID="25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f" Jul 29 15:05:18.577272 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577225 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 15:05:18.577536 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577517 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy-web" Jul 29 15:05:18.577536 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577536 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy-web" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577547 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy-thanos" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577556 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy-thanos" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577563 2562 scope.go:117] "RemoveContainer" containerID="21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577568 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="prometheus" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577635 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="prometheus" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577658 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="init-config-reloader" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577668 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="init-config-reloader" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577676 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577682 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577688 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="thanos-sidecar" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577694 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="thanos-sidecar" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577703 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="config-reloader" Jul 29 15:05:18.577749 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577709 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="config-reloader" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577794 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="thanos-sidecar" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577809 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="config-reloader" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577816 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy-web" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577821 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577828 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="prometheus" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577833 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="012e5799-a1c2-4f65-9c83-99c036996d11" containerName="kube-rbac-proxy-thanos" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:18.577866 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": container with ID starting with 21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e not found: ID does not exist" containerID="21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577890 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e"} err="failed to get container status \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": rpc error: code = NotFound desc = could not find container \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": container with ID starting with 21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e not found: ID does not exist" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.577906 2562 scope.go:117] "RemoveContainer" containerID="d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:18.578162 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": container with ID starting with d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9 not found: ID does not exist" containerID="d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.578190 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9"} err="failed to get container status \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": rpc error: code = NotFound desc = could not find container \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": container with ID starting with d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9 not found: ID does not exist" Jul 29 15:05:18.578285 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.578213 2562 scope.go:117] "RemoveContainer" containerID="4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568" Jul 29 15:05:18.578687 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:18.578484 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": container with ID starting with 4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568 not found: ID does not exist" containerID="4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568" Jul 29 15:05:18.578687 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.578501 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568"} err="failed to get container status \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": rpc error: code = NotFound desc = could not find container \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": container with ID starting with 4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568 not found: ID does not exist" Jul 29 15:05:18.578687 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.578514 2562 scope.go:117] "RemoveContainer" containerID="922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11" Jul 29 15:05:18.578801 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:18.578783 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": container with ID starting with 922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11 not found: ID does not exist" containerID="922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11" Jul 29 15:05:18.578833 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.578805 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11"} err="failed to get container status \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": rpc error: code = NotFound desc = could not find container \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": container with ID starting with 922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11 not found: ID does not exist" Jul 29 15:05:18.578833 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.578818 2562 scope.go:117] "RemoveContainer" containerID="3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e" Jul 29 15:05:18.579082 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:18.579061 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": container with ID starting with 3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e not found: ID does not exist" containerID="3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e" Jul 29 15:05:18.579150 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.579090 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e"} err="failed to get container status \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": rpc error: code = NotFound desc = could not find container \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": container with ID starting with 3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e not found: ID does not exist" Jul 29 15:05:18.579150 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.579109 2562 scope.go:117] "RemoveContainer" containerID="7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a" Jul 29 15:05:18.579497 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:18.579479 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": container with ID starting with 7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a not found: ID does not exist" containerID="7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a" Jul 29 15:05:18.579572 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.579505 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a"} err="failed to get container status \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": rpc error: code = NotFound desc = could not find container \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": container with ID starting with 7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a not found: ID does not exist" Jul 29 15:05:18.579572 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.579526 2562 scope.go:117] "RemoveContainer" containerID="25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f" Jul 29 15:05:18.579787 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:05:18.579773 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": container with ID starting with 25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f not found: ID does not exist" containerID="25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f" Jul 29 15:05:18.579830 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.579796 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f"} err="failed to get container status \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": rpc error: code = NotFound desc = could not find container \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": container with ID starting with 25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f not found: ID does not exist" Jul 29 15:05:18.579830 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.579809 2562 scope.go:117] "RemoveContainer" containerID="21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e" Jul 29 15:05:18.580073 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.580046 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e"} err="failed to get container status \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": rpc error: code = NotFound desc = could not find container \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": container with ID starting with 21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e not found: ID does not exist" Jul 29 15:05:18.580173 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.580074 2562 scope.go:117] "RemoveContainer" containerID="d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9" Jul 29 15:05:18.580405 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.580380 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9"} err="failed to get container status \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": rpc error: code = NotFound desc = could not find container \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": container with ID starting with d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9 not found: ID does not exist" Jul 29 15:05:18.580496 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.580405 2562 scope.go:117] "RemoveContainer" containerID="4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568" Jul 29 15:05:18.580765 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.580744 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568"} err="failed to get container status \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": rpc error: code = NotFound desc = could not find container \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": container with ID starting with 4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568 not found: ID does not exist" Jul 29 15:05:18.580828 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.580767 2562 scope.go:117] "RemoveContainer" containerID="922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11" Jul 29 15:05:18.581021 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.581000 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11"} err="failed to get container status \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": rpc error: code = NotFound desc = could not find container \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": container with ID starting with 922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11 not found: ID does not exist" Jul 29 15:05:18.581021 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.581020 2562 scope.go:117] "RemoveContainer" containerID="3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e" Jul 29 15:05:18.581319 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.581295 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e"} err="failed to get container status \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": rpc error: code = NotFound desc = could not find container \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": container with ID starting with 3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e not found: ID does not exist" Jul 29 15:05:18.581460 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.581322 2562 scope.go:117] "RemoveContainer" containerID="7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a" Jul 29 15:05:18.581573 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.581557 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a"} err="failed to get container status \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": rpc error: code = NotFound desc = could not find container \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": container with ID starting with 7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a not found: ID does not exist" Jul 29 15:05:18.581659 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.581573 2562 scope.go:117] "RemoveContainer" containerID="25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f" Jul 29 15:05:18.581834 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.581818 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f"} err="failed to get container status \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": rpc error: code = NotFound desc = could not find container \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": container with ID starting with 25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f not found: ID does not exist" Jul 29 15:05:18.581876 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.581837 2562 scope.go:117] "RemoveContainer" containerID="21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e" Jul 29 15:05:18.582041 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.582023 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e"} err="failed to get container status \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": rpc error: code = NotFound desc = could not find container \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": container with ID starting with 21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e not found: ID does not exist" Jul 29 15:05:18.582083 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.582043 2562 scope.go:117] "RemoveContainer" containerID="d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9" Jul 29 15:05:18.582233 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.582219 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9"} err="failed to get container status \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": rpc error: code = NotFound desc = could not find container \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": container with ID starting with d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9 not found: ID does not exist" Jul 29 15:05:18.582283 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.582233 2562 scope.go:117] "RemoveContainer" containerID="4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568" Jul 29 15:05:18.582436 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.582418 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568"} err="failed to get container status \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": rpc error: code = NotFound desc = could not find container \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": container with ID starting with 4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568 not found: ID does not exist" Jul 29 15:05:18.582474 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.582438 2562 scope.go:117] "RemoveContainer" containerID="922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11" Jul 29 15:05:18.582675 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.582660 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11"} err="failed to get container status \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": rpc error: code = NotFound desc = could not find container \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": container with ID starting with 922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11 not found: ID does not exist" Jul 29 15:05:18.582736 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.582675 2562 scope.go:117] "RemoveContainer" containerID="3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e" Jul 29 15:05:18.582896 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.582878 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e"} err="failed to get container status \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": rpc error: code = NotFound desc = could not find container \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": container with ID starting with 3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e not found: ID does not exist" Jul 29 15:05:18.582944 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.582897 2562 scope.go:117] "RemoveContainer" containerID="7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a" Jul 29 15:05:18.583086 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.583070 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a"} err="failed to get container status \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": rpc error: code = NotFound desc = could not find container \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": container with ID starting with 7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a not found: ID does not exist" Jul 29 15:05:18.583128 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.583088 2562 scope.go:117] "RemoveContainer" containerID="25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f" Jul 29 15:05:18.583172 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.583159 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.583581 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.583341 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f"} err="failed to get container status \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": rpc error: code = NotFound desc = could not find container \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": container with ID starting with 25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f not found: ID does not exist" Jul 29 15:05:18.583581 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.583363 2562 scope.go:117] "RemoveContainer" containerID="21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e" Jul 29 15:05:18.583734 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.583713 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e"} err="failed to get container status \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": rpc error: code = NotFound desc = could not find container \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": container with ID starting with 21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e not found: ID does not exist" Jul 29 15:05:18.583780 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.583735 2562 scope.go:117] "RemoveContainer" containerID="d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9" Jul 29 15:05:18.583975 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.583960 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9"} err="failed to get container status \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": rpc error: code = NotFound desc = could not find container \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": container with ID starting with d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9 not found: ID does not exist" Jul 29 15:05:18.584060 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.583976 2562 scope.go:117] "RemoveContainer" containerID="4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568" Jul 29 15:05:18.584197 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.584177 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568"} err="failed to get container status \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": rpc error: code = NotFound desc = could not find container \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": container with ID starting with 4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568 not found: ID does not exist" Jul 29 15:05:18.584242 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.584198 2562 scope.go:117] "RemoveContainer" containerID="922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11" Jul 29 15:05:18.584411 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.584395 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11"} err="failed to get container status \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": rpc error: code = NotFound desc = could not find container \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": container with ID starting with 922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11 not found: ID does not exist" Jul 29 15:05:18.584449 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.584411 2562 scope.go:117] "RemoveContainer" containerID="3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e" Jul 29 15:05:18.584708 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.584692 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e"} err="failed to get container status \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": rpc error: code = NotFound desc = could not find container \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": container with ID starting with 3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e not found: ID does not exist" Jul 29 15:05:18.584748 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.584709 2562 scope.go:117] "RemoveContainer" containerID="7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a" Jul 29 15:05:18.584906 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.584890 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a"} err="failed to get container status \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": rpc error: code = NotFound desc = could not find container \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": container with ID starting with 7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a not found: ID does not exist" Jul 29 15:05:18.584952 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.584907 2562 scope.go:117] "RemoveContainer" containerID="25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f" Jul 29 15:05:18.585180 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585145 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f"} err="failed to get container status \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": rpc error: code = NotFound desc = could not find container \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": container with ID starting with 25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f not found: ID does not exist" Jul 29 15:05:18.585180 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585168 2562 scope.go:117] "RemoveContainer" containerID="21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e" Jul 29 15:05:18.585645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585397 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 29 15:05:18.585645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585398 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 29 15:05:18.585645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585430 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e"} err="failed to get container status \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": rpc error: code = NotFound desc = could not find container \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": container with ID starting with 21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e not found: ID does not exist" Jul 29 15:05:18.585645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585455 2562 scope.go:117] "RemoveContainer" containerID="d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9" Jul 29 15:05:18.585645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585459 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 29 15:05:18.585645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585495 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 29 15:05:18.585645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585521 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 29 15:05:18.585645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585549 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 29 15:05:18.585645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585575 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 29 15:05:18.585645 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585575 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ao19e8qcnh73h\"" Jul 29 15:05:18.586214 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585695 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 29 15:05:18.586214 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585822 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 29 15:05:18.586214 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585871 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9"} err="failed to get container status \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": rpc error: code = NotFound desc = could not find container \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": container with ID starting with d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9 not found: ID does not exist" Jul 29 15:05:18.586214 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585890 2562 scope.go:117] "RemoveContainer" containerID="4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568" Jul 29 15:05:18.586214 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.585909 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 29 15:05:18.586214 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586108 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568"} err="failed to get container status \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": rpc error: code = NotFound desc = could not find container \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": container with ID starting with 4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568 not found: ID does not exist" Jul 29 15:05:18.586214 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586123 2562 scope.go:117] "RemoveContainer" containerID="922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11" Jul 29 15:05:18.586214 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586194 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 29 15:05:18.586214 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586219 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 29 15:05:18.586677 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586291 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jln5z\"" Jul 29 15:05:18.586677 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586219 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 29 15:05:18.586677 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586422 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11"} err="failed to get container status \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": rpc error: code = NotFound desc = could not find container \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": container with ID starting with 922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11 not found: ID does not exist" Jul 29 15:05:18.586677 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586443 2562 scope.go:117] "RemoveContainer" containerID="3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e" Jul 29 15:05:18.586865 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586698 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e"} err="failed to get container status \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": rpc error: code = NotFound desc = could not find container \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": container with ID starting with 3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e not found: ID does not exist" Jul 29 15:05:18.586865 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586711 2562 scope.go:117] "RemoveContainer" containerID="7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a" Jul 29 15:05:18.586950 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586930 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a"} err="failed to get container status \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": rpc error: code = NotFound desc = could not find container \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": container with ID starting with 7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a not found: ID does not exist" Jul 29 15:05:18.586950 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.586946 2562 scope.go:117] "RemoveContainer" containerID="25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f" Jul 29 15:05:18.587175 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.587155 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f"} err="failed to get container status \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": rpc error: code = NotFound desc = could not find container \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": container with ID starting with 25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f not found: ID does not exist" Jul 29 15:05:18.587175 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.587175 2562 scope.go:117] "RemoveContainer" containerID="21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e" Jul 29 15:05:18.587422 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.587401 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e"} err="failed to get container status \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": rpc error: code = NotFound desc = could not find container \"21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e\": container with ID starting with 21af27070af22a0a7df5dcaf39004a281c83938e94a56fb21737ce50fccd0d6e not found: ID does not exist" Jul 29 15:05:18.587486 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.587425 2562 scope.go:117] "RemoveContainer" containerID="d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9" Jul 29 15:05:18.587689 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.587664 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9"} err="failed to get container status \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": rpc error: code = NotFound desc = could not find container \"d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9\": container with ID starting with d86bae07eb1e0cd069683895b4c06b653785358e97c6ca7857affbb0fcc407f9 not found: ID does not exist" Jul 29 15:05:18.587689 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.587689 2562 scope.go:117] "RemoveContainer" containerID="4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568" Jul 29 15:05:18.588073 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.588037 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568"} err="failed to get container status \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": rpc error: code = NotFound desc = could not find container \"4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568\": container with ID starting with 4c6ae7cfe5ac682faebd58fe2cbec218b145236ccbe2695d4369aed7571e5568 not found: ID does not exist" Jul 29 15:05:18.588073 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.588061 2562 scope.go:117] "RemoveContainer" containerID="922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11" Jul 29 15:05:18.588897 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.588518 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11"} err="failed to get container status \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": rpc error: code = NotFound desc = could not find container \"922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11\": container with ID starting with 922056db6697b05d3aecb58609e85bcf4d1c729857f33f0d2a3115a12ef62a11 not found: ID does not exist" Jul 29 15:05:18.588897 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.588543 2562 scope.go:117] "RemoveContainer" containerID="3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e" Jul 29 15:05:18.588897 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.588809 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e"} err="failed to get container status \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": rpc error: code = NotFound desc = could not find container \"3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e\": container with ID starting with 3cc20071bc0c2145930a7b86cf57312034f910e9a6665dd9f3ae3aaa6dcaba6e not found: ID does not exist" Jul 29 15:05:18.588897 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.588831 2562 scope.go:117] "RemoveContainer" containerID="7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a" Jul 29 15:05:18.589157 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.589104 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a"} err="failed to get container status \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": rpc error: code = NotFound desc = could not find container \"7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a\": container with ID starting with 7c5f5fac3d9919af74f2c30ad49d31d283286418a11e42edbb614b5d456a298a not found: ID does not exist" Jul 29 15:05:18.589157 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.589124 2562 scope.go:117] "RemoveContainer" containerID="25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f" Jul 29 15:05:18.589477 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.589443 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f"} err="failed to get container status \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": rpc error: code = NotFound desc = could not find container \"25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f\": container with ID starting with 25a6c4d710d795706a98ba1d9e9012eb67b73a36f08d18078f918b809177238f not found: ID does not exist" Jul 29 15:05:18.589965 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.589943 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 29 15:05:18.592901 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.592881 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 29 15:05:18.597082 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.597064 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 15:05:18.633957 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.633916 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.633957 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.633959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.633988 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-config\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634087 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-web-config\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634127 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c6db72-a05c-4c29-a53a-afe5286efb16-config-out\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634261 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5bhn\" (UniqueName: \"kubernetes.io/projected/43c6db72-a05c-4c29-a53a-afe5286efb16-kube-api-access-v5bhn\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634337 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634360 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634388 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c6db72-a05c-4c29-a53a-afe5286efb16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634414 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634472 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634443 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634998 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634510 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634998 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634998 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634565 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634998 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634614 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634998 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634674 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.634998 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.634730 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.735949 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.735912 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.735949 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.735950 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736211 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.735978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-config\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736211 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736004 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736211 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736025 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-web-config\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736372 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736372 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736363 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c6db72-a05c-4c29-a53a-afe5286efb16-config-out\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736476 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736389 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736476 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5bhn\" (UniqueName: \"kubernetes.io/projected/43c6db72-a05c-4c29-a53a-afe5286efb16-kube-api-access-v5bhn\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736476 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736464 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736651 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736651 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c6db72-a05c-4c29-a53a-afe5286efb16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736651 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736552 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736651 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736651 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736648 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736674 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736706 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736755 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736789 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736802 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.736903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.736825 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.737251 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.737114 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.737327 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.737304 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.737475 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.737445 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.738166 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.737739 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.738363 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.738330 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.739085 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.739050 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.739396 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.739353 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.742020 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.741994 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c6db72-a05c-4c29-a53a-afe5286efb16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.743109 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.742933 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c6db72-a05c-4c29-a53a-afe5286efb16-config-out\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.743109 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.743066 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.743371 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.743120 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c6db72-a05c-4c29-a53a-afe5286efb16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.743371 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.743167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.743371 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.743315 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.743702 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.743521 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.743702 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.743586 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-config\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.743702 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.743586 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-web-config\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.744947 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.744926 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.745378 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.745344 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c6db72-a05c-4c29-a53a-afe5286efb16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.746222 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.746203 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5bhn\" (UniqueName: \"kubernetes.io/projected/43c6db72-a05c-4c29-a53a-afe5286efb16-kube-api-access-v5bhn\") pod \"prometheus-k8s-0\" (UID: \"43c6db72-a05c-4c29-a53a-afe5286efb16\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:18.894870 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:18.894783 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:19.049473 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:19.049442 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 15:05:19.052321 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:05:19.052295 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c6db72_a05c_4c29_a53a_afe5286efb16.slice/crio-5ffc8ef334974be59a530621cf29aa2f2048d6cc1190f83c86609565fa111dfa WatchSource:0}: Error finding container 5ffc8ef334974be59a530621cf29aa2f2048d6cc1190f83c86609565fa111dfa: Status 404 returned error can't find the container with id 5ffc8ef334974be59a530621cf29aa2f2048d6cc1190f83c86609565fa111dfa Jul 29 15:05:19.525761 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:19.525730 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c6db72-a05c-4c29-a53a-afe5286efb16","Type":"ContainerStarted","Data":"5ffc8ef334974be59a530621cf29aa2f2048d6cc1190f83c86609565fa111dfa"} Jul 29 15:05:20.537466 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:20.537425 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" event={"ID":"619ff693-e355-4bcc-91e9-90b26ef188c6","Type":"ContainerStarted","Data":"2d10d765b6e01cc747e0c82f20274efb9af3db717280e5d8bd6d0e40f45dc39c"} Jul 29 15:05:20.537951 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:20.537514 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" event={"ID":"619ff693-e355-4bcc-91e9-90b26ef188c6","Type":"ContainerStarted","Data":"9d7b70743583ee1a3d4e4ba0b36681f55ff1ed514b86b37df21bfdc82a422778"} Jul 29 15:05:20.537951 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:20.537542 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" event={"ID":"619ff693-e355-4bcc-91e9-90b26ef188c6","Type":"ContainerStarted","Data":"1203313cb6c8f05f5bd5019d8f4bdfc98ce618eef8ea387749ac188db362fa45"} Jul 29 15:05:20.539009 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:20.538990 2562 generic.go:356] "Generic (PLEG): container finished" podID="43c6db72-a05c-4c29-a53a-afe5286efb16" containerID="0524058628c45fbdfee705127b2969e6bdf70769f687e0ccef909c12c10e911e" exitCode=0 Jul 29 15:05:20.539053 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:20.539031 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c6db72-a05c-4c29-a53a-afe5286efb16","Type":"ContainerDied","Data":"0524058628c45fbdfee705127b2969e6bdf70769f687e0ccef909c12c10e911e"} Jul 29 15:05:20.576687 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:20.576638 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f4c7db59c-xflr6" podStartSLOduration=1.635026741 podStartE2EDuration="3.576621281s" totalImagesPullingTime="1.94159454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:05:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:05:20.575629867 +0000 UTC m=+244.484478249" watchObservedRunningTime="2026-07-29 15:05:20.576621281 +0000 UTC m=+244.485469661" Jul 29 15:05:21.546700 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:21.546666 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c6db72-a05c-4c29-a53a-afe5286efb16","Type":"ContainerStarted","Data":"ba3a25386267167e6c275fbd0934dea0499a93384a549eb1b03be8653133a4b4"} Jul 29 15:05:21.547194 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:21.546746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c6db72-a05c-4c29-a53a-afe5286efb16","Type":"ContainerStarted","Data":"b91c219bb655bf8c5e82a8979186e726f7974087f6e6bb21f34fdc8010e6340a"} Jul 29 15:05:21.547194 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:21.546786 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c6db72-a05c-4c29-a53a-afe5286efb16","Type":"ContainerStarted","Data":"5794611bc043e7b54c9ceb5b3ea082ef9866c51b9c6328fb5890cdd5dc440dce"} Jul 29 15:05:21.547194 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:21.546826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c6db72-a05c-4c29-a53a-afe5286efb16","Type":"ContainerStarted","Data":"d5cbcf28c4799146a693f3ed23df24982318112df2436c79cf2b009526d190f3"} Jul 29 15:05:21.547194 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:21.546869 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c6db72-a05c-4c29-a53a-afe5286efb16","Type":"ContainerStarted","Data":"accadba59e52b660def93678f1e83e8f5bfbb0df5b3396258788cc1cdfdd19fb"} Jul 29 15:05:21.547194 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:21.546907 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c6db72-a05c-4c29-a53a-afe5286efb16","Type":"ContainerStarted","Data":"503befbc2e8aa7670e01ce1f25bfbbd1a9546fccd3fc476961ac7541f636a6ff"} Jul 29 15:05:21.574908 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:21.574855 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.553962001 podStartE2EDuration="3.574836528s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.874527ms" isStatefulPod=false podCreationTimestamp="2026-07-29 15:05:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:05:21.574125016 +0000 UTC m=+245.482973397" watchObservedRunningTime="2026-07-29 15:05:21.574836528 +0000 UTC m=+245.483684909" Jul 29 15:05:21.684523 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:21.684491 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:21.783058 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:21.783025 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:23.171536 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:23.171498 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:23.895361 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:23.895311 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:05:27.513413 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:27.513367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:05:27.515810 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:27.515789 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c883406-1472-4688-819f-462bbb78df59-metrics-certs\") pod \"network-metrics-daemon-v4svt\" (UID: \"6c883406-1472-4688-819f-462bbb78df59\") " pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:05:27.769718 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:27.769641 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g48bm\"" Jul 29 15:05:27.775085 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:27.775055 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4svt" Jul 29 15:05:27.908714 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:27.908686 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v4svt"] Jul 29 15:05:27.911584 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:05:27.911554 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c883406_1472_4688_819f_462bbb78df59.slice/crio-85a99cf73b5d0b44e380a5124bb0d8185819bfb7fa26360c33232a46db6e05a9 WatchSource:0}: Error finding container 85a99cf73b5d0b44e380a5124bb0d8185819bfb7fa26360c33232a46db6e05a9: Status 404 returned error can't find the container with id 85a99cf73b5d0b44e380a5124bb0d8185819bfb7fa26360c33232a46db6e05a9 Jul 29 15:05:28.576301 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:28.576264 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v4svt" event={"ID":"6c883406-1472-4688-819f-462bbb78df59","Type":"ContainerStarted","Data":"85a99cf73b5d0b44e380a5124bb0d8185819bfb7fa26360c33232a46db6e05a9"} Jul 29 15:05:29.582495 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:29.582461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v4svt" event={"ID":"6c883406-1472-4688-819f-462bbb78df59","Type":"ContainerStarted","Data":"06a46104ca1d1a23bc23088084de81023919b78718c4d059892c38db2991da1b"} Jul 29 15:05:29.582891 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:29.582516 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v4svt" event={"ID":"6c883406-1472-4688-819f-462bbb78df59","Type":"ContainerStarted","Data":"d44c81f5464f735fd1945240b3d940fb8fdd4ed2ae9447c21b510147cc96f6ab"} Jul 29 15:05:29.602161 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:29.602112 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v4svt" podStartSLOduration=251.733947545 podStartE2EDuration="4m12.602097674s" totalImagesPullingTime="868.150129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:05:29.600397858 +0000 UTC m=+253.509246242" watchObservedRunningTime="2026-07-29 15:05:29.602097674 +0000 UTC m=+253.510946054" Jul 29 15:05:29.721376 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:29.721346 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:29.813932 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:29.813901 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:32.655348 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:32.655314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:32.756119 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:32.756085 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:53.152167 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:53.152082 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:53.216923 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:05:53.216887 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:16.484776 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:16.484751 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 29 15:06:18.895085 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:18.895044 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:06:18.911263 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:18.911237 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:06:19.789328 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:19.789298 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 15:06:27.405774 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:27.405746 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:27.471320 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:27.471292 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:39.370994 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:39.370952 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:39.433532 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:39.433491 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:46.167686 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:46.167648 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:46.267184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:46.267145 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:50.604845 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:50.604811 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:50.700370 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:50.700328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:53.845363 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:06:53.845313 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:05.043891 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:07:05.043857 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:05.144984 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:07:05.144952 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:05.219960 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:07:05.219917 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:08.244102 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:07:08.244008 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:08.342934 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:07:08.342903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:08.456321 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:07:08.456286 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:02.060150 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:02.060095 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:03.569344 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:03.569310 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:29.821984 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:29.821955 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:29.916950 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:29.916920 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:35.351294 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:35.351257 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:35.633184 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:35.633098 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:35.848789 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:35.848756 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:35.945294 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:35.945210 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:36.264810 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:36.264770 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:36.594443 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:36.594346 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:36.672264 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:36.672225 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:37.011392 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:37.011363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:37.122786 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:08:37.122752 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:37.841250 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:37.841207 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:52.890058 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:52.890020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:53.006717 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:53.006683 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:57.225097 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.225055 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:57.294761 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.294727 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-844cf74686-trcbc"] Jul 29 15:09:57.297934 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.297914 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-844cf74686-trcbc" Jul 29 15:09:57.300061 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.300038 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 29 15:09:57.300806 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.300788 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 29 15:09:57.300904 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.300788 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-6txk6\"" Jul 29 15:09:57.300904 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.300836 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 29 15:09:57.309794 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.309475 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-844cf74686-trcbc"] Jul 29 15:09:57.311346 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.311316 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:57.382626 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.382565 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:57.388871 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.388845 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cde77174-e480-4234-90fe-34bddf304faf-cert\") pod \"kserve-controller-manager-844cf74686-trcbc\" (UID: \"cde77174-e480-4234-90fe-34bddf304faf\") " pod="kserve/kserve-controller-manager-844cf74686-trcbc" Jul 29 15:09:57.389004 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.388962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5zt8\" (UniqueName: \"kubernetes.io/projected/cde77174-e480-4234-90fe-34bddf304faf-kube-api-access-k5zt8\") pod \"kserve-controller-manager-844cf74686-trcbc\" (UID: \"cde77174-e480-4234-90fe-34bddf304faf\") " pod="kserve/kserve-controller-manager-844cf74686-trcbc" Jul 29 15:09:57.483176 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.483069 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:57.490179 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.490147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5zt8\" (UniqueName: \"kubernetes.io/projected/cde77174-e480-4234-90fe-34bddf304faf-kube-api-access-k5zt8\") pod \"kserve-controller-manager-844cf74686-trcbc\" (UID: \"cde77174-e480-4234-90fe-34bddf304faf\") " pod="kserve/kserve-controller-manager-844cf74686-trcbc" Jul 29 15:09:57.490279 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.490195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cde77174-e480-4234-90fe-34bddf304faf-cert\") pod \"kserve-controller-manager-844cf74686-trcbc\" (UID: \"cde77174-e480-4234-90fe-34bddf304faf\") " pod="kserve/kserve-controller-manager-844cf74686-trcbc" Jul 29 15:09:57.492801 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.492779 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cde77174-e480-4234-90fe-34bddf304faf-cert\") pod \"kserve-controller-manager-844cf74686-trcbc\" (UID: \"cde77174-e480-4234-90fe-34bddf304faf\") " pod="kserve/kserve-controller-manager-844cf74686-trcbc" Jul 29 15:09:57.499223 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.499191 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5zt8\" (UniqueName: \"kubernetes.io/projected/cde77174-e480-4234-90fe-34bddf304faf-kube-api-access-k5zt8\") pod \"kserve-controller-manager-844cf74686-trcbc\" (UID: \"cde77174-e480-4234-90fe-34bddf304faf\") " pod="kserve/kserve-controller-manager-844cf74686-trcbc" Jul 29 15:09:57.583381 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.583333 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:57.610496 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.610464 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-844cf74686-trcbc" Jul 29 15:09:57.741011 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.740922 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-844cf74686-trcbc"] Jul 29 15:09:57.743747 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:09:57.743718 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde77174_e480_4234_90fe_34bddf304faf.slice/crio-4580ecf565eb0672e61637a883107ce3fa641063507ec0357b6268366aa18fee WatchSource:0}: Error finding container 4580ecf565eb0672e61637a883107ce3fa641063507ec0357b6268366aa18fee: Status 404 returned error can't find the container with id 4580ecf565eb0672e61637a883107ce3fa641063507ec0357b6268366aa18fee Jul 29 15:09:57.745299 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.745265 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 15:09:57.816333 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.816306 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:57.917260 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.917210 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:57.984084 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:57.984054 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:58.591124 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:09:58.591084 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-844cf74686-trcbc" event={"ID":"cde77174-e480-4234-90fe-34bddf304faf","Type":"ContainerStarted","Data":"4580ecf565eb0672e61637a883107ce3fa641063507ec0357b6268366aa18fee"} Jul 29 15:10:00.601450 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:00.601413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-844cf74686-trcbc" event={"ID":"cde77174-e480-4234-90fe-34bddf304faf","Type":"ContainerStarted","Data":"5625cc434c8ed7c889be70e5e89f1c60acf9ff94531fc4528433017eac71a59c"} Jul 29 15:10:00.601847 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:00.601545 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-844cf74686-trcbc" Jul 29 15:10:00.617697 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:00.617650 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-844cf74686-trcbc" podStartSLOduration=1.132674173 podStartE2EDuration="3.617636934s" totalImagesPullingTime="2.484962761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:10:00.616093981 +0000 UTC m=+524.524942364" watchObservedRunningTime="2026-07-29 15:10:00.617636934 +0000 UTC m=+524.526485320" Jul 29 15:10:09.307989 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:09.307890 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:09.368899 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:09.368855 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:09.692216 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:09.692123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:11.147582 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:11.147542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:22.384265 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:22.384232 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:31.613621 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:31.613569 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-844cf74686-trcbc" Jul 29 15:10:44.286088 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:44.286048 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:44.354719 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:10:44.354651 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:39.382494 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:11:39.382411 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:04.038888 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:14:04.038855 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:06.880557 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:14:06.880522 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:27.146568 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:14:27.146527 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:27.219165 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:14:27.219133 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:32.041043 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:14:32.041009 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:08.022307 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.022270 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-554db/must-gather-2khtf"] Jul 29 15:20:08.025657 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.025643 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:08.027920 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.027897 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-554db\"/\"openshift-service-ca.crt\"" Jul 29 15:20:08.028075 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.027971 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-554db\"/\"default-dockercfg-blg9q\"" Jul 29 15:20:08.028568 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.028551 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-554db\"/\"kube-root-ca.crt\"" Jul 29 15:20:08.042100 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.042080 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-554db/must-gather-2khtf"] Jul 29 15:20:08.078947 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.078930 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:08.121900 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.121874 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqsx2\" (UniqueName: \"kubernetes.io/projected/9efde71b-dc99-4c57-ba67-425fba401484-kube-api-access-pqsx2\") pod \"must-gather-2khtf\" (UID: \"9efde71b-dc99-4c57-ba67-425fba401484\") " pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:08.122021 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.121924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9efde71b-dc99-4c57-ba67-425fba401484-must-gather-output\") pod \"must-gather-2khtf\" (UID: \"9efde71b-dc99-4c57-ba67-425fba401484\") " pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:08.223283 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.223260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqsx2\" (UniqueName: \"kubernetes.io/projected/9efde71b-dc99-4c57-ba67-425fba401484-kube-api-access-pqsx2\") pod \"must-gather-2khtf\" (UID: \"9efde71b-dc99-4c57-ba67-425fba401484\") " pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:08.223386 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.223314 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9efde71b-dc99-4c57-ba67-425fba401484-must-gather-output\") pod \"must-gather-2khtf\" (UID: \"9efde71b-dc99-4c57-ba67-425fba401484\") " pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:08.223589 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.223576 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9efde71b-dc99-4c57-ba67-425fba401484-must-gather-output\") pod \"must-gather-2khtf\" (UID: \"9efde71b-dc99-4c57-ba67-425fba401484\") " pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:08.231641 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.231626 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqsx2\" (UniqueName: \"kubernetes.io/projected/9efde71b-dc99-4c57-ba67-425fba401484-kube-api-access-pqsx2\") pod \"must-gather-2khtf\" (UID: \"9efde71b-dc99-4c57-ba67-425fba401484\") " pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:08.347434 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.347368 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:08.470136 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.470112 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-554db/must-gather-2khtf"] Jul 29 15:20:08.472718 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:20:08.472693 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9efde71b_dc99_4c57_ba67_425fba401484.slice/crio-1642fb20c74c46f5d358663f5f2f33e5598927cdc44c808faabb7714ece25f42 WatchSource:0}: Error finding container 1642fb20c74c46f5d358663f5f2f33e5598927cdc44c808faabb7714ece25f42: Status 404 returned error can't find the container with id 1642fb20c74c46f5d358663f5f2f33e5598927cdc44c808faabb7714ece25f42 Jul 29 15:20:08.474524 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.474506 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 15:20:08.902314 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:08.902285 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-554db/must-gather-2khtf" event={"ID":"9efde71b-dc99-4c57-ba67-425fba401484","Type":"ContainerStarted","Data":"1642fb20c74c46f5d358663f5f2f33e5598927cdc44c808faabb7714ece25f42"} Jul 29 15:20:09.970481 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:09.970449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:15.937520 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:15.937479 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-554db/must-gather-2khtf" event={"ID":"9efde71b-dc99-4c57-ba67-425fba401484","Type":"ContainerStarted","Data":"caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115"} Jul 29 15:20:15.938265 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:15.938089 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-554db/must-gather-2khtf" event={"ID":"9efde71b-dc99-4c57-ba67-425fba401484","Type":"ContainerStarted","Data":"154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a"} Jul 29 15:20:15.956167 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:15.956113 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-554db/must-gather-2khtf" podStartSLOduration=1.277531848 podStartE2EDuration="7.956097859s" totalImagesPullingTime="6.678566011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:20:15.954696655 +0000 UTC m=+1139.863545037" watchObservedRunningTime="2026-07-29 15:20:15.956097859 +0000 UTC m=+1139.864946299" Jul 29 15:20:17.854761 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:17.854729 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:27.985066 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:27.985031 2562 generic.go:356] "Generic (PLEG): container finished" podID="9efde71b-dc99-4c57-ba67-425fba401484" containerID="154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a" exitCode=0 Jul 29 15:20:27.985405 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:27.985100 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-554db/must-gather-2khtf" event={"ID":"9efde71b-dc99-4c57-ba67-425fba401484","Type":"ContainerDied","Data":"154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a"} Jul 29 15:20:27.985445 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:27.985409 2562 scope.go:117] "RemoveContainer" containerID="154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a" Jul 29 15:20:28.941327 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:28.941301 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-554db_must-gather-2khtf_9efde71b-dc99-4c57-ba67-425fba401484/gather/0.log" Jul 29 15:20:29.288140 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:29.288101 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:29.440113 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:29.440086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:29.509775 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:29.509745 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:32.504833 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:32.504801 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qc8sd_e7b52735-d789-4074-9c97-2e5bdb782caa/global-pull-secret-syncer/0.log" Jul 29 15:20:32.553468 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:32.553447 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" Jul 29 15:20:32.695188 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:32.695163 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-104.ec2.internal_c19a4e2c03ee49097a3c2455ba7ff61f/haproxy/0.log" Jul 29 15:20:34.256897 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.256868 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-554db/must-gather-2khtf"] Jul 29 15:20:34.257397 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.257033 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-554db/must-gather-2khtf" podUID="9efde71b-dc99-4c57-ba67-425fba401484" containerName="copy" containerID="cri-o://caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115" gracePeriod=2 Jul 29 15:20:34.261233 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.260865 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-554db/must-gather-2khtf"] Jul 29 15:20:34.484398 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.484376 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-554db_must-gather-2khtf_9efde71b-dc99-4c57-ba67-425fba401484/copy/0.log" Jul 29 15:20:34.486188 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.486169 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:34.488126 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:20:34.488103 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-2khtf\" is forbidden: User \"system:node:ip-10-0-142-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-554db\": no relationship found between node 'ip-10-0-142-104.ec2.internal' and this object" podUID="9efde71b-dc99-4c57-ba67-425fba401484" pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:34.540561 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.540459 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9efde71b-dc99-4c57-ba67-425fba401484-must-gather-output\") pod \"9efde71b-dc99-4c57-ba67-425fba401484\" (UID: \"9efde71b-dc99-4c57-ba67-425fba401484\") " Jul 29 15:20:34.540561 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.540518 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqsx2\" (UniqueName: \"kubernetes.io/projected/9efde71b-dc99-4c57-ba67-425fba401484-kube-api-access-pqsx2\") pod \"9efde71b-dc99-4c57-ba67-425fba401484\" (UID: \"9efde71b-dc99-4c57-ba67-425fba401484\") " Jul 29 15:20:34.540895 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.540863 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9efde71b-dc99-4c57-ba67-425fba401484-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "9efde71b-dc99-4c57-ba67-425fba401484" (UID: "9efde71b-dc99-4c57-ba67-425fba401484"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 15:20:34.542886 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.542862 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9efde71b-dc99-4c57-ba67-425fba401484-kube-api-access-pqsx2" (OuterVolumeSpecName: "kube-api-access-pqsx2") pod "9efde71b-dc99-4c57-ba67-425fba401484" (UID: "9efde71b-dc99-4c57-ba67-425fba401484"). InnerVolumeSpecName "kube-api-access-pqsx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:20:34.568965 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.568937 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9efde71b-dc99-4c57-ba67-425fba401484" path="/var/lib/kubelet/pods/9efde71b-dc99-4c57-ba67-425fba401484/volumes" Jul 29 15:20:34.641903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.641872 2562 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9efde71b-dc99-4c57-ba67-425fba401484-must-gather-output\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:20:34.641903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.641897 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqsx2\" (UniqueName: \"kubernetes.io/projected/9efde71b-dc99-4c57-ba67-425fba401484-kube-api-access-pqsx2\") on node \"ip-10-0-142-104.ec2.internal\" DevicePath \"\"" Jul 29 15:20:34.836245 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.836138 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:34.891289 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:34.891253 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:35.013441 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:35.013417 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-554db_must-gather-2khtf_9efde71b-dc99-4c57-ba67-425fba401484/copy/0.log" Jul 29 15:20:35.013767 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:35.013745 2562 generic.go:356] "Generic (PLEG): container finished" podID="9efde71b-dc99-4c57-ba67-425fba401484" containerID="caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115" exitCode=143 Jul 29 15:20:35.013846 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:35.013795 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-554db/must-gather-2khtf" Jul 29 15:20:35.013846 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:35.013801 2562 scope.go:117] "RemoveContainer" containerID="caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115" Jul 29 15:20:35.021676 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:35.021651 2562 scope.go:117] "RemoveContainer" containerID="154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a" Jul 29 15:20:35.034424 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:35.034403 2562 scope.go:117] "RemoveContainer" containerID="caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115" Jul 29 15:20:35.034775 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:20:35.034754 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115\": container with ID starting with caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115 not found: ID does not exist" containerID="caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115" Jul 29 15:20:35.034861 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:35.034785 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115"} err="failed to get container status \"caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115\": rpc error: code = NotFound desc = could not find container \"caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115\": container with ID starting with caecbf4391ff79f3499f55f644269e017c8a85bf889cecc3252f2d0e0b313115 not found: ID does not exist" Jul 29 15:20:35.034861 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:35.034807 2562 scope.go:117] "RemoveContainer" containerID="154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a" Jul 29 15:20:35.035053 ip-10-0-142-104 kubenswrapper[2562]: E0729 15:20:35.035038 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a\": container with ID starting with 154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a not found: ID does not exist" containerID="154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a" Jul 29 15:20:35.035097 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:35.035056 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a"} err="failed to get container status \"154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a\": rpc error: code = NotFound desc = could not find container \"154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a\": container with ID starting with 154d82e44df15804060077b682bbbb7abe62858a7ee42959de48ef944f97606a not found: ID does not exist" Jul 29 15:20:36.817614 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:36.817564 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7a3a34b7-100b-47ae-86f2-95ae47689c26/alertmanager/0.log" Jul 29 15:20:36.842449 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:36.842408 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7a3a34b7-100b-47ae-86f2-95ae47689c26/config-reloader/0.log" Jul 29 15:20:36.868104 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:36.868076 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7a3a34b7-100b-47ae-86f2-95ae47689c26/kube-rbac-proxy-web/0.log" Jul 29 15:20:36.899330 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:36.899309 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7a3a34b7-100b-47ae-86f2-95ae47689c26/kube-rbac-proxy/0.log" Jul 29 15:20:36.925890 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:36.925866 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7a3a34b7-100b-47ae-86f2-95ae47689c26/kube-rbac-proxy-metric/0.log" Jul 29 15:20:36.951679 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:36.951647 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7a3a34b7-100b-47ae-86f2-95ae47689c26/prom-label-proxy/0.log" Jul 29 15:20:36.976618 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:36.976576 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7a3a34b7-100b-47ae-86f2-95ae47689c26/init-config-reloader/0.log" Jul 29 15:20:37.374351 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.374325 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wnppj_39569564-987e-4f48-a954-a76f1f1d5512/node-exporter/0.log" Jul 29 15:20:37.405545 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.405522 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wnppj_39569564-987e-4f48-a954-a76f1f1d5512/kube-rbac-proxy/0.log" Jul 29 15:20:37.430762 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.430738 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wnppj_39569564-987e-4f48-a954-a76f1f1d5512/init-textfile/0.log" Jul 29 15:20:37.575766 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.575692 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c6db72-a05c-4c29-a53a-afe5286efb16/prometheus/0.log" Jul 29 15:20:37.600049 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.600018 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c6db72-a05c-4c29-a53a-afe5286efb16/config-reloader/0.log" Jul 29 15:20:37.624853 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.624826 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c6db72-a05c-4c29-a53a-afe5286efb16/thanos-sidecar/0.log" Jul 29 15:20:37.650504 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.650482 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c6db72-a05c-4c29-a53a-afe5286efb16/kube-rbac-proxy-web/0.log" Jul 29 15:20:37.675138 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.675121 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c6db72-a05c-4c29-a53a-afe5286efb16/kube-rbac-proxy/0.log" Jul 29 15:20:37.699756 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.699728 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c6db72-a05c-4c29-a53a-afe5286efb16/kube-rbac-proxy-thanos/0.log" Jul 29 15:20:37.725206 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.725180 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c6db72-a05c-4c29-a53a-afe5286efb16/init-config-reloader/0.log" Jul 29 15:20:37.807175 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.807145 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-rz69n_99897e39-c00b-433e-b6b7-f879a2f58d83/prometheus-operator-admission-webhook/0.log" Jul 29 15:20:37.840207 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.840115 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f4c7db59c-xflr6_619ff693-e355-4bcc-91e9-90b26ef188c6/telemeter-client/0.log" Jul 29 15:20:37.866481 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.866445 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f4c7db59c-xflr6_619ff693-e355-4bcc-91e9-90b26ef188c6/reload/0.log" Jul 29 15:20:37.900607 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.900569 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f4c7db59c-xflr6_619ff693-e355-4bcc-91e9-90b26ef188c6/kube-rbac-proxy/0.log" Jul 29 15:20:37.935516 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.935492 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/thanos-query/0.log" Jul 29 15:20:37.965839 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.965806 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/kube-rbac-proxy-web/0.log" Jul 29 15:20:37.991012 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:37.990985 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/kube-rbac-proxy/0.log" Jul 29 15:20:38.017856 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:38.017825 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/prom-label-proxy/0.log" Jul 29 15:20:38.050568 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:38.050538 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/kube-rbac-proxy-rules/0.log" Jul 29 15:20:38.077283 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:38.077253 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d677d8db9-rcqvf_c6288dc9-fc9c-45eb-bded-aece30db7e75/kube-rbac-proxy-metrics/0.log" Jul 29 15:20:40.361032 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.360997 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8"] Jul 29 15:20:40.361526 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.361506 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9efde71b-dc99-4c57-ba67-425fba401484" containerName="gather" Jul 29 15:20:40.361634 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.361530 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9efde71b-dc99-4c57-ba67-425fba401484" containerName="gather" Jul 29 15:20:40.361634 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.361543 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9efde71b-dc99-4c57-ba67-425fba401484" containerName="copy" Jul 29 15:20:40.361634 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.361551 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9efde71b-dc99-4c57-ba67-425fba401484" containerName="copy" Jul 29 15:20:40.361794 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.361673 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9efde71b-dc99-4c57-ba67-425fba401484" containerName="copy" Jul 29 15:20:40.361794 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.361688 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9efde71b-dc99-4c57-ba67-425fba401484" containerName="gather" Jul 29 15:20:40.364970 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.364950 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.368059 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.368040 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rrj5m\"/\"openshift-service-ca.crt\"" Jul 29 15:20:40.368146 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.368119 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rrj5m\"/\"default-dockercfg-bmk4z\"" Jul 29 15:20:40.368720 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.368699 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rrj5m\"/\"kube-root-ca.crt\"" Jul 29 15:20:40.375787 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.375765 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8"] Jul 29 15:20:40.495683 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.495641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-sys\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.495683 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.495693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-lib-modules\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.495922 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.495721 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhkml\" (UniqueName: \"kubernetes.io/projected/1f097ba9-4824-4c11-b9cb-049197c1f5d9-kube-api-access-fhkml\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.495922 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.495821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-podres\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.495922 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.495849 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-proc\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.596910 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.596877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-sys\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.596910 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.596913 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-lib-modules\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.597132 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.596931 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhkml\" (UniqueName: \"kubernetes.io/projected/1f097ba9-4824-4c11-b9cb-049197c1f5d9-kube-api-access-fhkml\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.597132 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.597005 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-sys\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.597132 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.597045 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-podres\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.597132 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.597071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-proc\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.597132 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.597071 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-lib-modules\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.597345 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.597141 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-podres\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.597345 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.597164 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1f097ba9-4824-4c11-b9cb-049197c1f5d9-proc\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.605150 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.605121 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhkml\" (UniqueName: \"kubernetes.io/projected/1f097ba9-4824-4c11-b9cb-049197c1f5d9-kube-api-access-fhkml\") pod \"perf-node-gather-daemonset-cbjw8\" (UID: \"1f097ba9-4824-4c11-b9cb-049197c1f5d9\") " pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.674892 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.674796 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:40.809963 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:40.809940 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8"] Jul 29 15:20:40.812325 ip-10-0-142-104 kubenswrapper[2562]: W0729 15:20:40.812290 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1f097ba9_4824_4c11_b9cb_049197c1f5d9.slice/crio-2a2f144bdab080ffd468880c7f8f085332b79912b510a1247d403a8173073f29 WatchSource:0}: Error finding container 2a2f144bdab080ffd468880c7f8f085332b79912b510a1247d403a8173073f29: Status 404 returned error can't find the container with id 2a2f144bdab080ffd468880c7f8f085332b79912b510a1247d403a8173073f29 Jul 29 15:20:41.039902 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:41.039866 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" event={"ID":"1f097ba9-4824-4c11-b9cb-049197c1f5d9","Type":"ContainerStarted","Data":"8bab76644a165c4eca1a285c486e47747e0924d0df1ec68691eff3d2e2f24b32"} Jul 29 15:20:41.040074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:41.039915 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" event={"ID":"1f097ba9-4824-4c11-b9cb-049197c1f5d9","Type":"ContainerStarted","Data":"2a2f144bdab080ffd468880c7f8f085332b79912b510a1247d403a8173073f29"} Jul 29 15:20:41.040074 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:41.039985 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:41.070253 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:41.070194 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" podStartSLOduration=1.070176924 podStartE2EDuration="1.070176924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:20:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:20:41.068633377 +0000 UTC m=+1164.977481756" watchObservedRunningTime="2026-07-29 15:20:41.070176924 +0000 UTC m=+1164.979025384" Jul 29 15:20:42.018031 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:42.017997 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7dqs7_bfbfaa07-7732-4510-bfad-041401ba0c43/dns/0.log" Jul 29 15:20:42.042530 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:42.042506 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7dqs7_bfbfaa07-7732-4510-bfad-041401ba0c43/kube-rbac-proxy/0.log" Jul 29 15:20:42.207986 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:42.207953 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pp8jk_c20ebebb-b1b8-4d52-b617-b6faa3d4bf13/dns-node-resolver/0.log" Jul 29 15:20:42.701944 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:42.701915 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rthsh_30d268aa-dda6-492b-acd6-b14a838c5948/node-ca/0.log" Jul 29 15:20:43.788297 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:43.788270 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hv4wr_5318ca26-8284-482e-ab92-a3cb709fb40c/serve-healthcheck-canary/0.log" Jul 29 15:20:44.287202 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:44.287179 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w4784_d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514/kube-rbac-proxy/0.log" Jul 29 15:20:44.309122 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:44.309099 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w4784_d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514/exporter/0.log" Jul 29 15:20:44.330813 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:44.330786 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w4784_d45f0b1a-e6c0-4ee2-a0e7-754c2bf67514/extractor/0.log" Jul 29 15:20:46.526563 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:46.526527 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-844cf74686-trcbc_cde77174-e480-4234-90fe-34bddf304faf/manager/0.log" Jul 29 15:20:47.054088 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:47.054061 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rrj5m/perf-node-gather-daemonset-cbjw8" Jul 29 15:20:50.552677 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:50.552647 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-csrpv_c78120ec-62bc-4d4c-a4ab-b607b70ff0ea/migrator/0.log" Jul 29 15:20:50.579646 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:50.575065 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-csrpv_c78120ec-62bc-4d4c-a4ab-b607b70ff0ea/graceful-termination/0.log" Jul 29 15:20:51.772225 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:51.772154 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ftrpv_c9a1ce69-6788-4461-a3f8-e36f039aa692/kube-multus-additional-cni-plugins/0.log" Jul 29 15:20:51.796250 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:51.796232 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ftrpv_c9a1ce69-6788-4461-a3f8-e36f039aa692/egress-router-binary-copy/0.log" Jul 29 15:20:51.820104 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:51.820080 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ftrpv_c9a1ce69-6788-4461-a3f8-e36f039aa692/cni-plugins/0.log" Jul 29 15:20:51.842124 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:51.842103 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ftrpv_c9a1ce69-6788-4461-a3f8-e36f039aa692/bond-cni-plugin/0.log" Jul 29 15:20:51.863489 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:51.863462 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ftrpv_c9a1ce69-6788-4461-a3f8-e36f039aa692/routeoverride-cni/0.log" Jul 29 15:20:51.886335 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:51.886312 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ftrpv_c9a1ce69-6788-4461-a3f8-e36f039aa692/whereabouts-cni-bincopy/0.log" Jul 29 15:20:51.909193 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:51.909167 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ftrpv_c9a1ce69-6788-4461-a3f8-e36f039aa692/whereabouts-cni/0.log" Jul 29 15:20:52.281694 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:52.281667 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-npfvr_6800b94b-4d3d-4b5a-ab8b-68083bbd2520/kube-multus/0.log" Jul 29 15:20:52.348345 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:52.348319 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-v4svt_6c883406-1472-4688-819f-462bbb78df59/network-metrics-daemon/0.log" Jul 29 15:20:52.370265 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:52.370241 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-v4svt_6c883406-1472-4688-819f-462bbb78df59/kube-rbac-proxy/0.log" Jul 29 15:20:53.445403 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:53.445377 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cmhbw_4e087d29-d6fa-400e-a408-7d0a2cf8b6f3/ovn-controller/0.log" Jul 29 15:20:53.469952 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:53.469931 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cmhbw_4e087d29-d6fa-400e-a408-7d0a2cf8b6f3/ovn-acl-logging/0.log" Jul 29 15:20:53.498260 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:53.498238 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cmhbw_4e087d29-d6fa-400e-a408-7d0a2cf8b6f3/kube-rbac-proxy-node/0.log" Jul 29 15:20:53.523075 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:53.523053 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cmhbw_4e087d29-d6fa-400e-a408-7d0a2cf8b6f3/kube-rbac-proxy-ovn-metrics/0.log" Jul 29 15:20:53.546547 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:53.546525 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cmhbw_4e087d29-d6fa-400e-a408-7d0a2cf8b6f3/northd/0.log" Jul 29 15:20:53.569568 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:53.569545 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cmhbw_4e087d29-d6fa-400e-a408-7d0a2cf8b6f3/nbdb/0.log" Jul 29 15:20:53.593903 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:53.593878 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cmhbw_4e087d29-d6fa-400e-a408-7d0a2cf8b6f3/sbdb/0.log" Jul 29 15:20:53.637317 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:53.637286 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cmhbw_4e087d29-d6fa-400e-a408-7d0a2cf8b6f3/ovnkube-controller/0.log" Jul 29 15:20:54.806392 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:54.806368 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-2wrlr_eeb6b594-9641-4f08-875d-6c6d1eb68cb1/network-check-target-container/0.log" Jul 29 15:20:55.661278 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:55.661249 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-87r9f_fd305d57-301d-47e9-87cb-f664d2a9c6f1/iptables-alerter/0.log" Jul 29 15:20:56.426840 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:56.426803 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-lzwd6_24c52f73-7c5d-4452-96bc-4217e5341a27/tuned/0.log" Jul 29 15:20:56.488762 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:56.488716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:56.571342 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:56.571290 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:56.911175 ip-10-0-142-104 kubenswrapper[2562]: I0729 15:20:56.911096 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2brl7_22bad190-4a5f-47f7-a261-25ec14e59621/konnectivity-agent/0.log" limit=1024